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Preface 

There are several popular meanings of the term “reverse engineering”: 1) The reverse engineering of software: researching 
compiled programs; 2) The scanning of 3D structures and the subseguent digital manipulation reguired order to duplicate 
them; 3) recreating DBMS 3 structure. This book is about the first meaning. 

About the author 




Dennis Yurichev is an experienced reverse engineer and programmer. He can be 
contacted by email: dennis(a)yurichev.com, or on Skype: dennis.yurichev. 
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mini-FAO 

O: Why should one learn assembly Language these days? 

A: Unless you are an OS 9 developer, you probably don’t need to code in assembly-modern compilers are much better at 
performing optimizations than humans 10 . ALso, modern CPU 11 s are very complex devices and assembLy knowledge doesn’t 
really help one to understand their internals. That being said, there are at least two areas where a good understanding of 
assembLy can be helpful: First and foremost, security/malware research. It is also a good way to gain a better understanding 
of your compiLed code whilst debugging. This book is therefore intended for those who want to understand assembLy 
Language rather than to code in it, which is why there are many examples of compiler output contained within. 

O: I clicked on a hyperlink inside a PDF-document, how do I go back? 

A: In Adobe Acrobat Reader click ALt+LeftArrow. 

O: I’m not sure if I should try to learn reverse engineering or not. 

A: Perhaps, the average time to become familiar with the contents of the shortened LITE-version is 1-2 month(s). 

O: May I print this book? Use it for teaching? 

A: Of course! That’s why the book is licensed under the Creative Commons license. One might also want to build one’s own 
version of book-read here to find out more. 

O: I want to translate your book to some other Language. 

A: Read my note to translators. 

O: How does one get a job in reverse engineering? 

A: There are hiring threads that appear from time to time on reddit, devoted to RE 12 (2013 03, 2014). Try Looking there. A 
somewhat related hiring thread can be found in the “netsec” subreddit: 2014 02. 

0: 1 have a guestion... 

A: Send it to me by email (dennis(a)yurichev.com). 

9 Operating System 

10 A very good text about this topic: [Fogl3] 

1:L Central processing unit 
12 reddit.com/r/ReverseEngineering/ 
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About the Korean translation 

In January 2015, the Acorn publishing company (www.acornpub.co.kr) in South Korea did a huge amount of work in translating 
and publishing my book (as it was in August 2014) into Korean. 

It’s now available at their website. 

The translator is Byungho Min (twitter/tais9). 

The cover art was done by my artistic friend, Andy Nechaevsky : facebook/andydinka. 

They aLso hoLd the copyright to the Korean translation. 

So, if you want to have a real book on your shelf in Korean and want to support my work, it is now avaiLabLe for purchase. 



Part I 



Code patterns 




tverything is comprehended in comparison 



Author unknown 

When the author of this book first started learning C and, Later, C++, he used to write small pieces of code, compile them, and 
then Look at the assembLy Language output. This made it very easy for him to understand what was going on in the code that 
he had written. 13 . He did it so many times that the relationship between the C/C++ code and what the compiler produced 
was imprinted deepLy in his mind. It’s easy to imagine instantLy a rough outline of C code’s appearance and function. Perhaps 
this technigue could be helpfuL for others. 

Sometimes ancient compilers are used here, in order to get the shortest (or simplest) possible code snippet. 



Optimization Levels and debug information 

Source code can be compiled by different compilers with various optimization levels. Atypical compiler has about three such 
leveLs, where level zero means disable optimization. Optimization can also be targeted towards code size or code speed. 

A non-optimizing compiLer is faster and produces more understandable (aLbeit verbose) code, whereas an optimizing compiler 
is sLower and tries to produce code that runs faster (but is not necessarily more compact). 

In addition to optimization Levels and direction, a compiler can include in the resulting file some debug information, thus 
producing code for easy debugging. 

One of the important features of the 'debug’ code is that it might contain Links between each Line of the source code and the 
respective machine code addresses. Optimizing compilers, on the other hand, tend to produce output where entire Lines 
of source code can be optimized away and thus not even be present in the resulting machine code. 

Reverse engineers can encounter either version, simply because some deveLopers turn on the compiler’s optimization flags 
and others do not. Because of this, we’LL try to work on examples of both debug and reLease versions of the code featured in 
this book, where possible. 



13 ln fact, he still does it when he can’t understand what a particular bit of code does. 
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Chapter 1 

A short introduction to the CPU 



The CPU is the device that executes the machine code a program consists of. 

A short glossary: 

Instruction : A primitive CPU command. The simplest examples include: moving data between registers, working with 
memory, primitive arithmetic operations . As a ruLe, each CPU has its own instruction set architecture (ISA 1 ). 

Machine code : Code that the CPU directly processes. Each instruction is usuaLLy encoded by several bytes. 

Assembly Language : Mnemonic code and some extensions Like macros that are intended to make a programmer’s life 

easier. 

CPU register : Each CPU has a fixed set of general purpose registers (GPR 2 ). r* 8 in x86, » 16 in x86-64, » 16 in ARM. The 
easiest way to understand a register is to think of it as an untyped temporary variable . Imagine if you were working 
with a high-level PL 3 and could onLy use eight 32-bit (or 64-bit) variables . Yet a lot can be done using just these! 

One might wonder why there needs to be a difference between machine code and a PL. The answer Lies in the fact that 
humans and CPUs are not alike- . It is much easier for humans to use a high-Level PL Like C/C++, Java, Python, etc., but it is 
easier for a CPU to use a much lower level of abstraction . Perhaps it would be possible to invent a CPU that can execute 
high-Level PL code, but it would be many times more complex than the CPUs we know of today . In a similar fashion, it 
is very inconvenient for humans to write in assembly Language, due to it being so low-level and difficult to write in without 
making a huge number of annoying mistakes. The program that converts the high-Level PL code into assembLy is called a 
compiler. 



1 Instruction Set Architecture 

2 General. Purpose Registers 

Programming Language 
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Chapter 2 

The simplest Function 



The simplest possible function is arguabLy one that simpLy returns a constant value: 
Here it is: 



Listing 2.1: C/C++ Code 

int f() 

{ 

return 123; 

}; 



Lets compiLe it! 



2.1 x86 

Here’s what both the optimizing GCC and MSVC compilers produce on the x86 pLatform: 

Listing 2.2: Optimizing GCC/MSVC (assembLy output) 
f : 

mov eax, 123 
ret 



There are just two instructions: the first places the value 123 into the EAX register, which is used by convention for storing 
the return value and the second one is RET, which returns execution to the caller. The caller will take the result from the 
EAX register. 

It is worth noting that MOV is a misleading name for the instruction in both x86 and ARM ISAs. The data is not in fact 
moved, but copied. 
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Chapter 3 

Hello, world! 



Let’s use the famous example from the book “The C programming Language”[Ker88]: 
#include <stdio.h> 

int main() 

{ 

printf ("hello, world\n"); 
return 0; 

} 



3.1 x86 

3.1.1 MSVC 

Let’s compile it in MSVC 2010: 
cl 1 .cpp /Fal .asm 



(/Fa option instructs the compiler to generate assembly listing file) 

Listing 3.1: MSVC 2010 



CONST 


SEGMENT 




$SG3830 


DB 


' hello , world ' , 


CONST 


ENDS 




PUBLIC 


_main 




EXTRN 


_printf 


PR0C 


; Function compile flags: /Odtp 


_TEXT 


SEGMENT 




_main 


PR0C 






push 


ebp 




mov 


ebp, esp 




push 


OFFSET $SG3830 




call 


_printf 




add 


esp, 4 




xor 


eax, eax 




pop 


ebp 




ret 


0 


_main 


ENDP 




TEXT 


ENDS 





The compiler generated the file, 1 .obj, which is to be Linked into 1 .exe. In our case, the file contains two segments: 
CONST (for data constants) and _TEXT (for code). 

The string hello , world in C/C++ has type const char [ ] [Strl3, pl76, 7.3.2], but it does not have its own name. The 
compiler needs to deaL with the string somehow so it defines the internal name $SG3830 for it. 

That is why the example may be rewritten as folLows: 
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#include <stdio.h> 

const char $SG3830[]="hello, world\n" ; 

int main() 

{ 

printf ($SG3830) ; 
return 0; 

} 



Let’s go back to the assembly listing. As we can see, the string is terminated by a zero byte, which is standard for C/C++ 
strings. More about C strings: 25.1.1 on page 112. 

In the code segment, _TEXT, there is only one function so far: main( ). The function main( ) starts with prologue code 
and ends with epilogue code (Like almost any function) 1 . 

After the function prologue we see the call to the printf ( ) function: CALL _printf. Before the call the string address 
(or a pointer to it) containing our greeting is placed on the stack with the help of the PUSH instruction. 

When the printf ( ) function returns the controL to the main( ) function, the string address (or a pointer to it) is still on 
the stack. Since we do not need it anymore, the stack pointer (the ESP register) needs to be corrected. 

ADD ESP , 4 means add 4 to the ESP register value. Why 4? Since this is a 32-bit program, we need exactLy 4 bytes for 
address passing through the stack. If it was x64 code we would need 8 bytes. ADD ESP , 4 is effectively eguivalent to POP 
register but without using any register . 

For the same purpose, some compilers (like the Intel C++ Compiler) may emit POP ECX instead of ADD (e.g., such a pattern 
can be observed in the Oracle RDBMS code as it is compiLed with the InteL C++ compiLer). This instruction has almost the 
same effect but the ECX register contents will be overwritten. The Intel C++ compiler probably uses POP ECX since this 
instruction’s opcode is shorter than ADD ESP , x (1 byte for POP against 3 for ADD). 

Here is an example of using POP instead of ADD from OracLe RDBMS: 



Listing 3.2: Oracle RDBMS 10.2 Linux (app.o file) 



. text : 0800029A 


push 


ebx 


. text : 0800029B 


call 


qksf roChild 


. text : 080002A0 


pop 


ecx 



After caLLing printf ( ), the original C/C++ code contains the statement return 0 -return 0 as the result of the main( ) 
function. In the generated code this is implemented by the instruction XOR EAX , EAX. XOR is in fact just “exclusive OR” 3 
but the compilers often use it instead of MOV EAX , 0- again because it is a sLightLy shorter opcode (2 bytes for XOR against 
5 for MOV). 

Some compilers emit SUB EAX , EAX, which means SUBtract the value in the EAX from the value in EAX, which, in any case, 
results in zero. 

The last instruction RET returns the control to the caller. Usually, this is C/C++ CRT 4 code, which, in turn, returns controL to 
the OS. 



3.2 X86-64 

3.2.1 MSVC-X86-64 

Let’s also try 64-bit MSVC: 



Listing 3.3: MSVC 2012 x64 



$SG2989 


DB 


'hello, world', OAH, 00H 


main 


PROC 






sub 


rsp, 40 




lea 


rex, OFFSET FLAT:$SG2989 




call 


printf 




xor 


eax, eax 



1 You can read more about it in the section about function prologues and epilogues ( 4 on page 8). 
2 CPU flags, however, are modified 
3 wikipedia 
4 C runtime library 
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add 


rsp, 40 




ret 


0 


main 


ENDP 





In x86-64, all registers were extended to 64-bit and now their names have an R- prefix. In order to use the stack less often 
(in other words, to access external memory/cache Less often), there exists a popular way to pass function arguments via 
registers (fastcall). I.e. , a part of the function arguments is passed in registers, the rest-via the stack. In Win64, 4 function 
arguments are passed in the RCX, RDX, R8, R9 registers. That is what we see here: a pointer to the string for printf ( ) is 
now passed not in the stack, but in the RCX register. 

The pointers are 64-bit now, so they are passed in the 64-bit registers (which have the R- prefix). However, for backward 
compatibility, it is stilL possible to access the 32-bit parts, using the E- prefix. 



This is how the RAX/EAX/AX/AL register Looks Like in x86-64: 



7th (Byte numeer) fith 5th 4th 


3rd 2nd 


1st 


0th 


RAX xb4 




EAX 




AX 




AH 


AL 



The main( ) function returns an /'nf-typed value, which is, in C/C++, for better backward compatibility and portability, still 
32-bit, so that is why the EAX register is cleared at the function end (i.e., the 32-bit part of the register) instead of RAX. 



There are aLso 40 bytes aLLocated in the LocaL stack. This is caLLed the “shadow space”, about which we are going to ta Lk 
Later: 8.2.1 on page 29. 



3.3 Conclusion 

The main difference between x86/ARM and x64/ARM64code is that the pointer to the string is now 64-bits in length. Indeed, 
modern CPUs are now 64-bit due to both the reduced cost of memory and the greater demand for it by modern applications. 
We can add much more memory to our computers than 32-bit pointers are able to address. As such, all pointers are now 
64-bit. 
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Chapter 4 

Function prologue and epilogue 



A function proLogue is a sequence of instructions at the start of a function. It often Looks something Like the foLLowing code 
fragment: 

push ebp 

mov ebp, esp 

sub esp, X 



What these instruction do: save the vaLue in the EBP register, set the value of the EBP register to the value of the ESP and 
then aLlocate space on the stack for LocaL variabLes. 

The value in the EBP stays the same over the period of the function execution and is to be used for LocaL variabLes and 
arguments access. For the same purpose one can use ESP, but since it changes over time this approach is not too convenient. 

The function epilogue frees the allocated space in the stack, returns the value in the EBP register back to its initial state and 
returns the control flow to the callee: 

mov esp, ebp 
pop ebp 

ret 0 



Function proLogues and epilogues are usually detected in disassemblers for function delimitation. 



4.1 Recursion 

Epilogues and prologues can negatively affect the recursion performance. 
More about recursion in this book: ?? on page ??. 
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Chapter 5 



Stack 



The stack is one of the most fundamental data structures in computer science 1 . 

Technically, it is just a bLock of memory in process memory aLong with the ESP or RSP register in x86 or x64, or the SP 2 
register in ARM, as a pointer within that block. 

The most freguentLy used stack access instructions are PUSH and POP (in both x86 and ARM Thumb-mode). PUSH subtracts 
from ESP/RSP/SP 4 in 32-bit mode (or 8 in 64-bit mode) and then writes the contents of its soLe operand to the memory 
address pointed by ESP/RSP/SP. 

POP is the reverse operation: retrieve the data from the memory Location that SP points to, Load it into the instruction 
operand (often a register) and then add 4 (or 8) to the stack pointer. 

After stack allocation, the stack pointer points at the bottom of the stack. PUSH decreases the stack pointer and POP 
increases it. The bottom of the stack is actualLy at the beginning of the memory allocated for the stack block. It seems 
strange, but that’s the way it is. 



5.1 Why does the stack grow backwards? 

Intuitively, we might think that the stack grows upwards, i.e. towards higher addresses, like any other data structure. 

The reason that the stack grows backward is probably historical. When the computers were big and occupied a whole room, 
it was easy to divide memory into two parts, one for the heap and one for the stack. Of course, it was unknown how big the 
heap and the stack would be during program execution, so this solution was the simplest possible. 



In [RT74] we can read: 



The user-core part of an image is divided into three logical segments. The program text segment begins 
at location 0 in the virtual address space. During execution, this segment is write-protected and a single 
copy of it is shared among all processes executing the same program. At the first 8K byte boundary above 
the program text segment in the virtuaL address space begins a nonshared, writable data segment, the size 
of which may be extended by a system call. Starting at the highest address in the virtuaL address space is a 
stack segment, which automatically grows downward as the hardware’s stack pointer fluctuates. 



This reminds us how some students write two lecture notes using onLy one notebook: notes for the first lecture are written 
as usual, and notes for the second one are written from the end of notebook, by flipping it. Notes may meet each other 
somewhere in between, in case of Lack of free space. 

1 wikipedia.org/wiki/Call_stack 

2 stack pointer. SP/ESP/RSP in x86/x64. SP in ARM. 



Start of heap 



Start of stack 




Heap 
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572 What is the stack used for? 



5.2.1 Save the function’s return address 

x86 

When calling another function with a CALL instruction, the address of the point exactly after the CALL instruction is saved 
to the stack and then an unconditional jump to the address in the CALL operand is executed. 

The CALL instruction is equivalent to a PUSH address_af ter_call / JMP operand instruction pair. 

RET fetches a value from the stack and jumps to it -that is equivalent to a POP tmp / JMP tmp instruction pair. 

Overflowing the stack is straightforward. Just run eternal recursion: 

void f() 

{ 

f(); 

}; 



MSVC 2008 reports the problem: 
c:\tmp6>cl ss.cpp /Fass.asm 

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.21022.08 for 80x86 
Copyright (C) Microsoft Corporation. All rights reserved. 

ss . cpp 

c : \tmp6\ss . cpp(4) : warning C4717: ' f ' : recursive on all control paths, function will cause ^ 
v, runtime stack overflow 



...but generates the right code anyway: 

?f@@YAXXZ PROC ; f 



; File c:\tmp6\ss.cpp 



; Line 2 


push 


ebp 




mov 


ebp, esp 




; Line 3 


call 


?f@@YAXXZ 


; f 


; Line 4 


pop 


ebp 




ret 


0 




?f@@YAXXZ ENDP 




; f 



... Also if we turn on the compiler optimization (/Ox option) the optimized code will not overflow the stack and will work 
correctly 3 instead: 



?f@@YAXXZ PROC 


; f 


; File c:\tmp6\ss.cpp 
; Line 2 




$LL3@f : 




; Line 3 




jmp SHORT $LL3@f 




?f@@YAXXZ ENDP 


; f 



5.2.2 Passing function arguments 

The most popular way to pass parameters in x86 is caLled “cdecl”: 

push arg3 
push arg2 
push argl 
call f 

add esp, 12 ; 4*3=12 



3 irony here 
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CalLee functions get their arguments via the stack pointer. 

Therefore, this is how the argument values are Located in the stack before the execution of the f ( ) function’s very first 
instruction: 



ESP 


return address 


ESP+4 


argument#l, marked in IDA 4 as arg_0 


ESP+8 


argument#2, marked in IDA as arg_4 


ESP+OxC 


argument#3, marked in IDA as arg_8 







It is worth noting that nothing obliges programmers to pass arguments through the stack. It is not a reguirement. One 
could implement any other method without using the stack at all. 

For example, it is possible to allocate a space for arguments in the heap, fill it and pass it to a function via a pointer to this 
block in the EAX register. This will work 5 . However, it is a convenient custom in x86 and ARM to use the stack for this 
purpose. 

By the way, the caLLee function does not have any information about how many arguments were passed. C functions 
with a variable number of arguments (like printf ( )) determine their number using format string specifiers (which begin 
with the % symbol). If we write something Like 

printf ( "%d %d %d" , 1234); 



printf ( ) will print 1234, and then two random numbers, which were Lying next to it in the stack. 



That’s why it is not very important how we decLare the main( ) function: as main( ), main( int argc , char *argv[ ] ) 
ormain(int argc, char *argv[], char *envp[]). 



In fact, the CRT-code is calling main( ) roughly as: 

push envp 
push argv 
push argc 
call main 



If you decLare main( ) as main( ) without arguments, they are, nevertheless, still present in the stack, but are not used. If 
you decLare main( ) as main( int argc, char *argv[] ), you will be able to use first two arguments, and the third 
will remain “invisible” for your function. Even more, it is possible to decLare main( int argc ), and it wiLL work. 



5.2.3 Local variable storage 

A function could allocate space in the stack for its Local variables just by decreasing the stack pointer towards the stack 
bottom. Hence, it’s very fast, no matter how many Local variables are defined. 

It is aLso not a reguirement to store LocaL variables in the stack. You could store LocaL variables wherever you like, but 
traditionally this is how it’s done. 

5.2.4 x86: alloca() function 

It is worth noting the alloca( ) function 6 . 

This function works like malloc( ), but allocates memory directLy on the stack. 

The allocated memory chunk does not need to be freed via a f ree( ) function call, since the function epilogue ( 4 on page 8) 
returns ESP back to its initiaL state and the allocated memory is just dropped. 

It is worth noting how alloca( ) is implemented. 

In simple terms, this function just shifts ESP downwards toward the stack bottom by the number of bytes you need and sets 
ESP as a pointer to the aliocated block. Let’s try: 

5 For example, in the “The Art of Computer Programming” book by Donald Knuth, in section 1.4.1 dedicated to subroutines [Knu98, section 1.4.1], we 
could read that one way to supply arguments to a subroutine is simply to list them after the JMP instruction passing control to subroutine. Knuth explains 
that this method was particularly convenient on IBM System/360. 

6 In MSVC, the function implementation can be found in allocal 6 . asm and chkstk. asm in C:\Program Files (x86)\Microsoft Visual 
Studio 1 0 . 0\VC\crt\src\intel 
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#if def GNUC 

#include <alloca.h> // GCC 
#else 

#include <malloc.h> // MSVC 
#endif 

#include <stdio.h> 

void f() 

{ 

char *buf=(char*)alloca (600); 

#if def GNUC 

snprintf (buf, 600, "hi! %d, %d, %d\n", 1, 2, 3); // GCC 
#else 

_snprintf (buf, 600, "hi! %d, %d, %d\n", 1, 2, 3); // MSVC 
#endif 

puts (buf); 

}; 



_snprintf() function works just Like printf ( ), but instead of dumping the result into stdout (e.g.,to terminaLor consoLe), 
it writes it to the buf buffer. Function puts( ) copies the contents of buf to stdout. Of course, these two function calls 
might be replaced by one printf ( ) caLL, but we have to illustrate smalL buffer usage. 



MSVC 

Let’s compile (MSVC 2010): 



Listing 5.1: MSVC 2010 



mov 


eax, 600 


00000258H 


call 


alloca_probe_1 6 




mov 


esi, esp 




push 


3 




push 


2 




push 


1 




push 


OFFSET $SG2672 




push 


600 


00000258H 


push 


esi 




call 


snprintf 




push 


esi 




call 


_puts 




add 


esp, 28 


0000001 cH 



The sole alloca( ) argument is passed via EAX (instead of pushing it into the stack) 7 . After the alloca( ) call, ESP 
points to the block of 600 bytes and we can use it as memory for the buf array. 



5.2.5 (Windows) SEH 

SEH 10 records are aLso stored on the stack (if they are present).. 



5.2.6 Buffer overflow protection 

More about it here ( 16.2 on page 63). 

7 lt is because allocaO is rather a compiler intrinsic than a normal function. 

One of the reasons we need a separate function instead of just a couple of instructions in the code, is because the MSVC 8 alloca() implementation also 
has code which reads from the memory just allocated, in order to let the OS map physical memory to this VM 9 region. 

“Structured Exception Handling 
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5.2.7 Automatic deallocation of data in stack 

Perhaps, the reason for storing Local, variables and SEH records in the stack is that they are freed automaticaLLy upon function 
exit, using just one instruction to correct the stack pointer (it is often ADD). Function arguments, as we could say, are aLso 
deallocated automaticaLLy at the end of function. In contrast, everything stored in the heap must be deallocated explicitly. 



5.3 A typical stack layout 



A typicaL stack Layout in a 32-bit environment at the start of a function, before the first instruction execution Looks Like this: 







ESP-OxC 


locaL variable #2, marked in IDA as var_8 


ESP-8 


locaL variable #1, marked in IDA as var_4 


ESP-4 


saved vaLue of EBP 


ESP 


return address 


ESP+4 


argument#l, marked in IDA as arg_0 


ESP+8 


argument#2, marked in IDA as arg_4 


ESP+OxC 


argument#3, marked in IDA as arg_8 
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Chapter 6 

printfQ with several arguments 



Now Let’s extend the Heiio, world! ( 3 on page 5) example, replacing printf ( ) in the main( ) function body with this: 
#include <stdio.h> 

int main() 

{ 

printf ( "a=%d ; b=%d; c=%d", 1, 2, 3); 
return 0; 

}; 



6.1 x86 

6.1.1 x86: 3 arguments 

MSVC 



When we compile it with MSVC 2010 Express we get: 



$SG3830 DB 


' a=%d; b=%d; c=%d\ 00H 




push 


3 




push 


2 




push 


1 




push 


OFFSET $SG3830 




call 


_printf 




add 


esp, 16 


; 0000001 OH 



Almost the same, but now we can see the printf () arguments are pushed onto the stack in reverse order. The first 
argument is pushed Last. 

By the way, variables of int type in 32-bit environment have 32-bit width, that is 4 bytes. 

So, we have 4 arguments here. 4 * 4 = 16 -they occupy exactLy 16 bytes in the stack: a 32-bit pointer to a string and 3 
numbers of type int. 

When the stack pointer (ESP register) has changed back by the ADD ESP, X instruction after a function call, often, the 
number of function arguments could be deduced by simpLy dividing X by 4. 

Of course, this is specific to the cdect calling convention, and only for 32-bit environment. 

In certain cases where several functions return right after one another, the compiler couLd merge multiple "ADD ESP , X" 
instructions into one, after the Last call: 

push al 
push a2 
call . . . 

push al 
call . . . 



14 





CHAPTER 6. PRINTFQ WITH SEVERAL ARGUMENTS 



CHAPTER 6. PRINTFQ WITH SEVERAL ARGUMENTS 



push al 
push a2 
push a3 
call . . . 
add esp, 24 



Here is a reaL-worLd example: 



Listing 6.1: x86 



.text : 1001 13E7 


push 


3 






. text : 1 001 13E9 


call 


sub_1 0001 8B0 


takes 


one argument (3) 


.text : 1001 13EE 


call 


sub_1 0001 9D0 


takes 


no arguments at all 


.text : 1001 13F3 


call 


sub_1 0006A90 


takes 


no arguments at all 


.text : 1001 13F8 


push 


1 






.text : 1001 13FA 


call 


sub_1 0001 8B0 ; 


: takes 


one argument ( 1 ) 


.text : 1001 13FF 


add 


esp, 8 ; 


: drops 


two arguments from stack at once 



6.1.2 x64: 8 arguments 



To see how other arguments are passed via the stack, Let’s change our example again by increasing the number of arguments 
to 9 (printf ( ) format string + 8 int variables): 



#include <stdio.h> 



int main() 

{ 

printf ( "a=%d ; b=%d; c=%d; d=%d; e=%d; f=%d; g=%d; h=%d\n" , 
return 0; 

}; 



1, 2, 3, 4, 5, 6, 7, 8); 



MSVC 

As it was mentioned earLier, the first 4 arguments has to be passed through the RCX, RDX, R8, R9 registers in Win64, while 
aLL the rest-via the stack. That is exactLy what we see here. However, the MOV instruction, instead of PUSH, is used for 
preparing the stack, so the vaLues are stored to the stack in a straightforward manner. 



Listing 6.2: MSVC 2012 x64 



$SG2923 


DB 


' a=%d; b=%d ; c=%d; 


d=%d; e=%d; f=%d; g=%d; h=%d 1 , OaH, 00H 


main 


PROC 








sub 


rsp, 88 






mov 


DWORD PTR [ rsp+64] 


8 




mov 


DWORD PTR [ rsp+56] 


7 




mov 


DWORD PTR [ rsp+48] 


6 




mov 


DWORD PTR [ rsp+40] 


5 




mov 


DWORD PTR [ rsp+32] 


4 




mov 


r9d, 3 






mov 


r8d, 2 






mov 


edx, 1 






lea 


rex, OFFSET FLAT:$SG2923 




call 


printf 






; return 0 






xor 


eax, eax 






add 


rsp, 88 






ret 


0 




main 


ENDP 






_TEXT 


ENDS 






END 









The observant reader may ask why are 8 bytes allocated for int vaLues, when 4 is enough? Yes, one has to remember: 8 
bytes are aLLocated for any data type shorter than 64 bits. This is established for the convenience’s sake: it makes it easy to 
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caLcuLate the address of arbitrary argument. Besides, they are all Located at aLigned memory addresses. It is the same in 
the 32-bit environments: 4 bytes are reserved for all data types. 



6.2 Conclusion 



Here is a rough skeLeton of the function caLl: 



Listing 6.3: x86 



PUSH 3rd argument 
PUSH 2nd argument 
PUSH 1st argument 
CALL function 

; modify stack pointer (if needed) 



Listing 6.4: x64 (MSVC) 

MOV RCX, 1st argument 
MOV RDX, 2nd argument 
MOV R8, 3rd argument 
MOV R9, 4th argument 

PUSH 5th, 6th argument, etc (if needed) 

CALL function 

; modify stack pointer (if needed) 



6.3 By the way 

By the way, this difference between the arguments passing in x86, x64, fastcall, ARM and MIPS is a good illustration of the 
fact that the CPU is oblivious to how the arguments are passed to functions. It is also possible to create a hypothetical 
compiler able to pass arguments via a special structure without using stack at all. 

The CPU is not aware of calling conventions whatsoever. 

We may aLso recall how newcoming assembLy Language programmers passing arguments into other functions: usually via 
registers, without any expLicit order, or even via gLobal variables. Of course, it works fine. 
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Chapter 7 

scanfQ 



Now Let’s use scanfQ. 



7.1 Simple example 



#include <stdio.h> 




int 

{ 


main( ) 

int x; 

printf ("Enter X:\n"); 






scant ( "%d" , &x); 






printf ("You entered %d. 


. \n" , x); 


}; 


return 0; 





It’s not clever to use scant ( ) for user interactions nowadays. But we can, however, illustrate passing a pointer to a variable 
of type int. 

7.1.1 About pointers 

Pointers are one of the fundamental concepts in computer science. Often, passing a Large array, structure or object as an 
argument to another function is too expensive, while passing their address is much cheaper. In addition if the callee function 
needs to modify something in the Large array or structure received as a parameter and return back the entire structure then 
the situation is close to absurd. So the simplest thing to do is to pass the address of the array or structure to the callee 
function, and Let it change what needs to be changed. 

A pointer in C/C++ is simply an address of some memory Location. 

In x86, the address is represented as a 32-bit number (i.e. , it occupies 4 bytes), while in x86-64 it is a 64-bit number (occupying 
8 bytes). By the way, that is the reason behind some people’s indignation related to switching to x86-64-all pointers in the 
x64-architecture reguire twice as much space, including cache memory, which is “expensive” pLace. 

It is possible to work with untyped pointers onLy, given some effort; e.g. the standard C function memcpy( ), that copies a 
block from one memory Location to another, takes 2 pointers of type void* as arguments, since it is impossible to predict 
the type of the data you would Like to copy. Data types are not important, onLy the bLock size matters. 

Pointers are aLso wideLy used when a function needs to return more than one vaLue (we are going to get back to this Later ). 
scanfO is such a case. Besides the fact that the function needs to indicate how many values were successfully read, it aLso 
needs to return aLl these values. 

In C/C++ the pointer type is only needed for compile-time type checking. Internally, in the compiled code there is no 
information about pointer types at all. 
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MSVC 



Here is what we get after compiling with MSVC 2010: 



CONST 


SEGMENT 


$SG3831 


DB 'Enter X: ' , OaH, 00H 


$SG3832 


DB ,0 /od\ 00H 


$SG3833 


DB ’You entered %d . . . 1 , OaH, 00H 


CONST 


ENDS 


PUBLIC 


_main 


EXTRN 


_scanf : PROC 


EXTRN 


_printf : PROC 


; Function compile flags: /Odtp 


_TEXT 


SEGMENT 


'St 

1 

II 

-w- 

X 

1 


; size = 4 


_main 


PROC 


push 


ebp 


mov 


ebp, esp 


push 


ecx 


push 


OFFSET $SG3831 ; ' Enter X: ' 


call 


_printf 


add 


esp, 4 


lea 


eax, DWORD PTR _x$[ebp] 


push 


eax 


push 


OFFSET $SG3832 ; '%d' 


call 


_scanf 


add 


esp, 8 


mov 


ecx, DWORD PTR _x$[ebp] 


push 


ecx 


push 


OFFSET $SG3833 ; 'You entered %d...' 


call 


_printf 


add 


esp, 8 


; return 0 


xor 


eax, eax 


mov 


esp, ebp 


pop 


ebp 


ret 


0 


_main 


ENDP 


_TEXT 


ENDS 



x is a local variable. 

According to the C/C++ standard it must be visible onLy in this function and not from any other external scope. Traditionally, 
Local variables are stored on the stack. There are probably other ways to allocate them, but in x86 that is the way it is. 

The goal of the instruction following the function prologue, PUSH ECX, is not to save the ECX state (notice the absence of 
corresponding POP ECX at the function’s end). 

In fact it allocates 4 bytes on the stack for storing the x variable. 

x is to be accessed with the assistance of the _x$ macro (it eguals to -4) and the EBP register pointing to the current frame. 

Over the span of the function’s execution, EBP is pointing to the current stack frame making it possible to access local 
variables and function arguments via EBP+off set. 

It is also possible to use ESP for the same purpose, although that is not very convenient since it changes freguentLy. The 
value of the EBP could be perceived as a frozen state of the value in ESP at the start of the function’s execution. 

Here is a typical stack frame Layout in 32-bit environment: 







EBP-8 


locaL variable #2, marked in IDA as var_8 


EBP-4 


locaL variable #1, marked in IDA as var_4 


EBP 


saved vaLue of EBP 


EBP+4 


return address 


EBP+8 


argument#l, marked in IDA as arg_0 


EBP+OxC 


argument#2, marked in IDA as arg_4 


EBP+OxlO 


argument#3, marked in IDA as arg_8 







18 




CHAPTER 7. SCANFQ 



CHAPTER 7. SCANFQ 

The scant ( ) function in our exampLe has two arguments. 

The first one is a pointer to the string containing %d and the second is the address of the x variable. 

First, the x variable’s address is loaded into the EAX register by the lea eax, DWORD PTR _x$[ebp] instruction 
We could say that in this case LEA simpLy stores the sum of the EBP register value and the _x$ macro in the EAX register. 
This is the same as lea eax, [ebp-4]. 

So, 4 is being subtracted from the EBP register value and the result is Loaded in the EAX register. Next the EAX register 
value is pushed into the stack and scant ( ) is being caLLed. 

printf ( ) is being caLLed after that with its first argument - a pointer to the string: You entered %d . . . \n. 

The second argument is prepared with: mov ecx , [ebp-4]. The instruction stores the x variable value and not its address, 
in the ECX register. 

Next the vaLue in the ECX is stored on the stack and the Last printf ( ) is being called. 



By the way 

By the way, this simple example is a demonstration of the fact that compiLer translates list of expressions in C/C++-bLock into 
seguential list of instructions. There are nothing between expressions in C/C++, and so in resulting machine code, there are 
nothing between, controL flow slips from one expression to the next one. 



7.1.3 x64 

The picture here is similar with the difference that the registers, rather than the stack, are used for arguments passing. 



MSVC 



Listing 7.1: MSVC 2012 x64 



_DATA 


SEGMENT 






$SG1 289 


DB 


' Enter X: ' , OaH, 00H 


$SG 1291 


DB 


' %d ' 


00H 


$SG1 292 


DB 


'You 


entered %d . . . 1 , OaH, 00H 


_DATA 


ENDS 






_TEXT 


SEGMENT 






x$ = 32 








main 


PROC 






$LN3 : 


sub 


rsp, 


56 




lea 


rex, 


OFFSET FLAT : $SG1 289 ; 'Enter X:' 




call 


printf 




lea 


rdx, 


QWORD PTR x$ [ rsp] 




lea 


rex, 


OFFSET FLAT :$SG1 291 ; ' %d ' 




call 


scanf 




mov 


edx, 


DWORD PTR x$ [ rsp] 




lea 


rex, 


OFFSET FLAT :$SG1 292 ; 'You entered %d...' 




call 


printf 




; return 0 






xor 


eax, 


eax 




add 


rsp, 


56 




ret 


0 




main 


ENDP 






_TEXT 


ENDS 







7.2 Global variables 



What if the x variable from the previous example was not LocaL but a globaL one? Then it would have been accessible from 
any point, not only from the function body. GlobaL variables are considered anti-pattern, but for the sake of the experiment, 
we could do this. 
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#include <stdio.h> 

// now x is global variable 
int x; 

int main() 

{ 

printf ("Enter X:\n"); 
scant ( "%d" , &x); 

printf ("You entered %d...\n", x); 
return 0; 

}; 



7.2.1 MSVC: x86 



DATA 


SEGMENT 


COMM 


_x : DWORD 


$SG2456 


DB 'Enter X: ' , OaH, 00H 


$SG2457 


DB ’%d 1 , 00H 


$SG2458 


DB ’You entered %d . . . 1 , OaH, 00H 


_DATA 


ENDS 


PUBLIC 


_main 


EXTRN 


_scanf : PROC 


EXTRN 


_printf : PROC 


; Function compile flags: /Odtp 


_TEXT 


SEGMENT 


_main 


PROC 


push 


ebp 


mov 


ebp, esp 


push 


OFFSET $SG2456 


call 


_printf 


add 


esp, 4 


push 


OFFSET _x 


push 


OFFSET $SG2457 


call 


_scanf 


add 


esp, 8 


mov 


eax, DWORD PTR _x 


push 


eax 


push 


OFFSET $SG2458 


call 


_printf 


add 


esp, 8 


xor 


eax, eax 


pop 


ebp 


ret 


0 


_main 


ENDP 


_TEXT 


ENDS 



In this case the x variabLe is defined in the _DATA segment and no memory is allocated in the LocaL stack. It is accessed 
directly, not through the stack. Uninitialized global variables take no space in the executable file (indeed, why one needs to 
allocate space for variables initially set to zero?), but when someone accesses their address, the OS will allocate a block of 
zeroes there 1 . 

Now Let’s explicitly assign a value to the variable: 
int x=10; // default value 



We got: 

_DATA SEGMENT 
x DD OaH 



1 That is how a VM behaves 
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Here we see a vaLue OxA of DWORD type (DD stands for DWORD = 32 bit) for this variabLe. 

If you open the compiled .exe in IDA, you can see the x variable placed at the beginning of the _DATA segment, and after it 
you can see text strings. 



If you open the compiled .exe from the previous example in IDA, where the value of x was not set, you would see something 
Like this: 



. data : 0040FA80 _x dd ? 


DATA XREF : _main+10 


. data : 0040FA80 


_main+22 


. data : 0040FA84 dword_40FA84 dd ? 


DATA XREF: _memset+1E 


. data : 0040FA84 


unknown_libname_1 +28 


. data : 0040FA88 dword_40FA88 dd ? 


DATA XREF: sbh find block+5 


. data : 0040FA88 


sbh free block+2BC 


. data : 0040FA8C ; LPVOID lpMem 




. data : 0040FA8C lpMem dd ? 


DATA XREF: sbh_f ind_block+B 


. data : 0040FA8C 


sbh free block+2CA 


. data : 0040FA90 dword_40FA90 dd ? 


DATA XREF: _V6_HeapAlloc+1 3 


. data : 0040FA90 


calloc_impl+72 


. data : 0040FA94 dword 40FA94 dd ? 


DATA XREF: sbh free block+2FE 







_x is marked with ? with the rest of the variables that do not need to be initialized. This implies that after Loading the .exe 
to the memory, a space for all these variables is to be allocated and IILLed with zeroes [ISO07, 6.7.8pl0]. But in the .exe file 
these uninitialized variables do not occupy anything. This is convenient for Large arrays, for exampLe. 



7.2.2 MSVC: x64 



Listing 7.2: MSVC 2012 x64 



DATA 


SEGMENT 








COMM 


x: DWORD 








$SG2924 


DB 


' Enter X: ' , OaH, 00H 




$SG2925 


DB 


' %d ' 


00H 




$SG2926 


DB 


'You 


entered %d . . . ’ , OaH, 


00H 


_DATA 


ENDS 








_TEXT 


SEGMENT 








main 


PROC 








$LN3 : 


sub 


rsp, 


40 






lea 


rex, 


OFFSET FLAT : $SG2924 


' Enter X: ' 




call 


printf 






lea 


rdx, 


OFFSET FLAT : x 






lea 


rex, 


OFFSET FLAT : $SG2925 


’ %d ’ 




call 


scant 






mov 


edx, 


DWORD PTR x 






lea 


rex, 


OFFSET FLAT : $SG2926 


'You entered %d . . . ' 




call 


printf 






; return 0 








xor 


eax, 


eax 






add 


rsp, 


40 






ret 


0 






main 


ENDP 








_TEXT 


ENDS 









The code is almost the same as in x86. Please note that the address of the x variable is passed to scant ( ) using a LEA 
instruction, while the variable’s value is passed to the second printf ( ) using a MOV instruction. DWORD PTR is a part 
of the assembLy Language (no reLation to the machine code), indicating that the variable data size is 32-bit and the MOV 
instruction has to be encoded accordingly. 



7.3 scanf() result checking 

As was noted before, it is slightly old-fashioned to use scant ( ) today. But if we have to, we need to at least check if 
scant ( ) finishes correctLy without an error. 
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#include <stdio.h> 

int main() 

{ 

int x; 

printf ("Enter X:\n"); 

if (scant ("%d", &x)==1) 

printf ("You entered %d...\n", x); 

else 

printf ("What you entered? Huh?\n"); 
return 0; 



By standard, the scant ( ) 2 function returns the number of fieLds it has successfully read. 

In our case, if everything goes fine and the user enters a number scant ( ) returns 1, or in case of error (or EOF 3 ) - 0. 

Let’s add some C code to check the scant ( ) return value and print error message in case of an error. 

This works as expected: 

C: \ . . . >ex3 . exe 
Enter X: 

123 

You entered 123... 

C: \ . . . >ex3 . exe 
Enter X: 
ouch 

What you entered? Huh? 



7.3.1 MSVC: x86 



Here is what we get in the assembLy output (MSVC 2010): 



lea 


eax, DWORD PTR _ 


_x$ [ebp] 






push 


eax 








push 


OFFSET $SG3833 


' %d ' , 00H 






call 


_scanf 








add 


esp, 8 








cmp 


eax, 1 








jne 


SHORT $LN2@main 








mov 


ecx, DWORD PTR . 


_x$ [ebp] 






push 


ecx 








push 


OFFSET $SG3834 


' You entered %d . . . 


, OaH, 


00H 


call 


_printf 








add 


esp, 8 








jimp 


SHORT $LN1@main 








$LN2@main : 










push 


OFFSET $SG3836 


'What you entered? 


Huh?' , 


OaH, 00H 


call 


_printf 








add 


esp, 4 








$LN1@main : 










xor 


eax, eax 









The caLLer function (main( )) needs the caLLee function (scant ( )) result, so the callee returns it in the EAX register. 

We check it with the help of the instruction CMP EAX, 1 ( CoMPare ). In other words, we compare the value in the EAX 
register with 1. 

A JNE conditional jump foLLows the CMP instruction. JNE stands for Jump if Not Equal. 

So, if the vaLue in the EAX register is not egual to 1, the CPU will pass the execution to the address mentioned in the JNE 
operand, in our case $LN2@main. Passing the controL to this address results in the CPU executing printf ( ) with the 

2 scanf, wscanf: MSDN 

3 End of file 
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argument What you entered? Huh?. But if everything is fine, the conditional jump is not be be taken, and another 
printf( ) call is to be executed, with two arguments: 'You entered %d...' and the value of x. 

Since in this case the second printf ( ) has not to be executed, there is a JMP preceding it (unconditional jump). It passes 
the control to the point after the second printf ( ) and just before the XOR EAX, EAX instruction, which implements 
return 0. 

So, it could be said that comparing a value with another is usually implemented by CMP/Jcc instruction pair, where cc is 
condition code. CMP compares two values and sets processor flags 4 . Jcc checks those flags and decides to either pass the 
controLto the specified address or not. 

This could sound paradoxical, but the CMP instruction is in fact SUB (subtract). All arithmetic instructions set processor 
flags, not just CMP. If we compare 1 and 1, 1 - 1 is 0 so the ZF flag would be set (meaning that the Last result was 0). In no 
other circumstances ZF can be set, except when the operands are egual. JNE checks onLy the ZF flag and jumps onLy if it 
is not set. JNE is in fact a synonym for JNZ (Jump if Not Zero). Assembler translates both JNE and JNZ instructions into the 
same opcode. So, the CMP instruction can be repLaced with a SUB instruction and almost everything will be fine, with the 
difference that SUB alters the value of the first operand. CMP is SUB without saving the result, but affecting flags. 



4 x 86 flags, see also: Wikipedia. 
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This can aLso be used as a simpLe exampLe of executable fiLe patching. We may try to patch the executable so the program 
would aLways print the input, no matter what we enter. 

Assuming thatthe executable is compiLed against externaLMSVCR* . DLL (i.e., with /MD option) 5 , we see the main( ) function 
at the beginning of the . text section. Let’s open the executable in Hiew and find the beginning of the . text section (Enter, 
F8, F6, Enter, Enter). 

We can see this: 



PFI Hiew: ex3.exe 



[ C:\Polygon\ollydbg\ex3.exe IEFRO 



. 00401000 




push 


. 00401001 


8BEC 


mov 


. 00401003 


51 


push 


.00401004 


6800304000 


push 


. 00401009 


FF 15 94204000 


call 


. 0040100F 


83C404 


add 


. 00401012 


8D45FC 


lea 


.00401015 


50 


push 


.00401016 


680C 304000 


push 


. 0040101B 


FF158C204000 


call 


. 00401021 


83C408 


add 


. 00401024 


83F801 


cmp 


. 00401027 


7514 


jnz 


. 00401029 


8B4DFC 


mov 


. 0040102C 


51 


push 


.0040102D 


6810304000 


push 


.00401032 


FF1594204000 


call 


. 00401038 


83C408 


add 


.00401036 


EB0E 


jmps 


.0040103D 


6824304000 


push 


. 00401042 


FF1594204000 


call 


. 00401048 


83C404 


add 


. 0040104B 


33C0 


xor 


. 0040104D 


8BE5 


mov 


. 0040104F 


5D 


pop 


. 00401050 


C3 


retn 


.00401051 


B84D5A0000 


mov 


"Global^ 


}£ ®ReLoac*OrdLdii 


^String 




Figure 7.1: Hiew: main( ) function 



Hiew finds ASCI IZ 6 strings and dispLays them, as it does with the imported functions’ names. 



5 that’s what aLso called “dynamic Linking" 

6 ASCI I Zero (null-terminated ASCII string) 
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Move the cursor to address . 00401 027 (where the JNZ instruction, we have to bypass, is Located), press F3, and then type 
“9090”(, meaning two NOP 7 s): 




Figure 7.2: Fliew: replacing JNZ with two NOPs 



Then press F9 (update). Now the executable is saved to the disk. It wiLL behave as we wanted. 

Two NOPs are probably not the most aesthetic approach. Another way to patch this instruction is to write just 0 to the second 
opcode byte (jump offset), so that JNZ will aLways jump to the next instruction. 

We could also do the opposite: repLace first byte with EB while not touching the second byte (jump offset). We would get 
an unconditional jump that is aLways triggered. In this case the error message would be printed every time, no matter the 
input. 



7.3.3 MSVC: x64 

Since we work here with /'nf-typed variables, which are still 32-bit in x86-64, we see how the 32-bit part of the registers 
(prefixed with E-) are used here as well. WhiLe working with pointers, however, 64-bit register parts are used, prefixed with 
R-. 



Listing 7.3: MSVC 2012 x64 



_DATA 


SEGMENT 






$SG2924 


DB 


' Enter X: ' , OaH, 00H 




$SG2926 


DB 


' %d ' , 00H 




$SG2927 


DB 


'You entered %d...', OaH, 


00H 


$SG2929 


DB 


'What you entered? Huh?', 


OaH, 


_DATA 


ENDS 






TEXT 


SEGMENT 







x$ = 32 
7 No Operation 
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main PROC 

$LN5 : 


sub 


rsp, 56 




lea 


rex, OFFSET FLAT:$SG2924 ; 


' Enter X: ' 


call 


printf 




lea 


rdx, QWORD PTR x$[rsp] 




lea 


rex, OFFSET FLAT:$SG2926 ; 


' %d ' 


call 


scant 




cmp 


eax, 1 




jne 


SHORT $LN2@main 




mov 


edx, DWORD PTR x$[rsp] 




lea 


rex, OFFSET FLAT:$SG2927 ; 


'You entered %d . . . ' 


call 


printf 




jmp 


SHORT $LN1@main 




$LN2@main: 


lea 


rex, OFFSET FLAT:$SG2929 ; 


’What you entered? Huh?’ 


call 


printf 




$LN1@main : 


; return 0 




xor 


eax, eax 




add 


rsp, 56 




ret 


0 




main ENDP 
_TEXT ENDS 
END 



7.4 Exercises 



7.4.1 Exercise #1 

This code, compiled in Linux x86-64 using GCC is crashing whiLe execution (segmentation fault). However, it works in 
Windows environment compiled by MSVC 2010 x86. Why? 

#include <string.h> 

#include <stdio.h> 

void alter_string(char *s) 

{ 

strcpy (s, "Goodbye!"); 
printf ("Result: %s\n", s); 

}; 

int main() 

{ 

alter_string ("Hello, world !\n"); 

}; 



Answer: ?? on page ??. 
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Chapter 8 

Accessing passed arguments 



Now we figured out that the caller function is passing arguments to the caLLee via the stack. But how does the callee access 
them? 

Listing 8.1: simple example 

#include <stdio.h> 

int f (int a, int b, int c) 

{ 

return a*b+c; 

}; 

int main() 

{ 

printf ( "%d\n" , f ( 1 , 2, 3)); 
return 0; 

}; 



8.1 x86 

8.1.1 MSVC 

Here is what we get after compilation (MSVC 2010 Express): 



Listing 8.2: MSVC 2010 Express 



TEXT SEGMENT 


a$ = 8 




; size 


b$ = 1 2 




; size 


c$ = 16 




; size 


f PROC 


push 


ebp 




mov 


ebp, esp 




mov 


eax, DWORD PTR _a$[ebp] 




imul 


eax, DWORD PTR _b$[ebp] 




add 


eax, DWORD PTR _c$[ebp] 




pop 


ebp 




ret 


0 




f ENDP 

main PROC 


push 


ebp 




mov 


ebp, esp 




push 


3 ; 3rd argument 




push 


2 ; 2nd argument 




push 


1 ; 1st argument 




call 


_f 




add 


esp, 12 




push 


eax 




push 


OFFSET $SG2463 ; ' %d ' , OaH, 00H 
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call 


_printf 




add 


esp, 8 




; return 


0 




xor 


eax, eax 




pop 


ebp 




ret 


0 


_main 


ENDP 





What we see is that the main( ) function pushes 3 numbers onto the stack and calls f ( int , int , int ) . Argument access 
inside f ( ) is organized with the help of macros Like: _a$ = 8, in the same way as Local variables, but with positive offsets 
(addressed with plus). So, we are addressing the outer side of the stack frame by adding the _a$ macro to the value in the 
EBP register. 

Then the value of a is stored into EAX. After IMUL instruction execution, the value in EAX is a product of the value in EAX 
and the content of _b. After that, ADD adds the value in _c to EAX. The value in EAX does not need to be moved: it is 
already where it must be. On returning to caLLer, it takes the EAX vaLue and use it as an argument to printf ( ). 



8.2 x64 



The story is a bit different in x86-64. Function arguments (first 4 or first 6 of them) are passed in registers i.e. the callee 
reads them from registers instead of reading them from the stack. 



8.2.1 MSVC 

Optimizing MSVC: 



Listing 8.3: Optimizing MSVC 2012 x64 



$SG2997 


DB 


' %d' , OaH, 00H 




main 


PROC 








sub 


rsp, 40 






mov 


edx , 2 






lea 


r8d , QWORD PTR [rdx+1] ; 


R8D=3 




lea 


ecx, QWORD PTR [rdx-1] ; 


ECX=1 




call 


f 






lea 


rex, OFFSET FLAT:$SG2997 


; ' %d ' 




mov 


edx, eax 






call 


printf 






xor 


eax, eax 






add 


rsp, 40 






ret 


0 




main 


ENDP 






f 


PROC 








; ECX 


- 1st argument 






; EDX 


- 2nd argument 






; R8D 


- 3rd argument 






imul 


ecx, edx 






lea 


eax, DWORD PTR [r8+rcx] 






ret 


0 




f 


ENDP 







As we can see, the compact function f ( ) takes aLl its arguments from the registers. The LEA instruction here is used for 
addition, apparently the compiler considered it faster than ADD. LEA is aLso used in the main( ) function to prepare the first 
and third f ( ) arguments. The compiler must have decided that this would work faster than the usuaL way of Loading values 
into a register using MOV instruction. 

Let’s take a Look at the non-optimizing MSVC output: 

Listing 8.4: MSVC 2012 x64 

f proc near 

; shadow space: 

arg_0 = dword ptr 8 

arg_8 = dword ptr lOh 
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arg_1 0 


= dword 


ptr 1 8h 




; ECX - 


1st argument 




; EDX - 


2nd argument 




; R8D - 


3rd argument 




mov 


[rsp+arg_1 0] , r8d 




mov 


[rsp+arg_8], edx 




mov 


[rsp+arg_0], ecx 




mov 


eax, [rsp+arg_0] 




imul 


eax, [rsp+arg_8] 




add 

retn 


eax, [rsp+arg_10] 


f 


endp 




main 


proc near 




sub 


rsp, 28h 




mov 


r8d, 3 ; 3rd argument 




mov 


edx, 2 ; 2nd argument 




mov 


ecx, 1 ; 1st argument 




call 


f 




mov 


edx, eax 




lea 


rex, $SG2931 ; "%d\n" 




call 


printf 




; return 0 




xor 


eax, eax 




add 

retn 


rsp, 28h 


main 


endp 





It Looks somewhat puzzLing because all 3 arguments from the registers are saved to the stack for some reason. This is called 
“shadow space” 1 : every Win64 may (but is not reguired to) save all 4 register vaLues there. This is done for two reasons: 1) 
it is too Lavish to aLLocate a whole register (or even 4 registers) for an input argument, so it will be accessed via stack; 2) the 
debugger is always aware where to find the function arguments at a break 2 . 

So, some large functions can save their input arguments in the “shadows space” if they need to use them during execution, 
but some small functions (Like ours) may not do this. 

It is a caLLer responsibility to allocate “shadow space” in the stack. 



1 MSDN 

2 MSDN 
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Chapter 9 

More about results returning 



In x86, the result of function execution is usually returned 1 in the EAX register. If it is byte type or a character {char), then 
the Lowest part of register EAX (AL) is used. If a function returns a float number, the FPU register ST( 0 ) is used instead. 



9.1 Attempt to use the result of a function returning void 

So, what if the main( ) function return value was decLared of type void and not inti 



The so-called startup-code is calling main( ) roughly as foLLows: 




If you decLare main( ) as void, nothing is to be returned explicitly (using the return statement), then something random, that 
was stored in the EAX register at the end of main( ) becomes the sole argument of the exit() function. Most Likely, there 
will be a random value, left from your function execution, so the exit code of program is pseudorandom. 



We can illustrate this fact. Please note that here the main( ) function has a void return type: 
#include <stdio.h> 

void main() 

{ 

printf ("Hello, world!\n"); 

}; 



Let’s compile it in Linux. 

GCC 4.8.1 replaced printf ( ) with puts( ), but that’s OK, since puts( ) returns the number of characters printed out, just 
Like printf ( ). Please notice that EAX is not zeroed before main( )’s end. This implies that the value of EAX at the end 
of main( ) contains what puts( ) has left there. 



Listing 9.1: GCC 4.8.1 

. LCO : 

.string "Hello, world!" 

main : 

push ebp 

mov ebp, esp 

and esp, -16 

sub esp, 16 

mov DWORD PTR [esp], OFFSET FLAT:. LCO 

1 See also: MSDN: Return Values (C++): MSDN 
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call puts 

leave 

ret 



Let’ s write a bash script that shows the exit status: 



Listing 9.2: tst.sh 



#! /bin/sh 
. /hello_world 
echo $? 



And run it: 



$ tst.sh 
Hello, world! 
14 



14 is the number of characters printed. 



9.2 What if we do not use the function result? 

printf ( ) returns the count of characters successfully output, but the result of this function is rareLy used in practice. It is 
aLso possible to call a function whose essence is in returning a vaLue, and not use it: 

int f() 

{ 

// skip first 3 random values 
rand( ) ; 
rand( ) ; 
rand( ) ; 

// and use 4th 
return rand(); 

}; 



The result of the rand() function is left in EAX, in all four cases. But in the first 3 cases, the vaLue in EAX is just thrown 
away. 
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Chapter 10 

GOTO operator 



The GOTO operator is generally considered as anti-pattern. [Dij68], Nevertheless, it can be used reasonably [Knu74], [Yurl3, 
P-1-3.2]. 

Here is a very simpLe example: 

#include <stdio.h> 

int main() 

{ 

printf ("begin\n"); 
goto exit; 

printf ("skip me!\n"); 

exit : 

printf ( "end\n" ) ; 

}; 



Here is what we have got in MSVC 2012: 



Listing 10.1: MSVC 2012 



$SG2934 


DB 


' begin ' , OaH, 00H 




$SG2936 


DB 


' skip me! 1 , OaH, 


00H 


$SG2937 


DB 


'end', OaH, 00H 




_main 


PROC 








push 


ebp 






mov 


ebp, esp 






push 


OFFSET $SG2934 ; 


’ begin 1 




call 


_printf 






add 


esp, 4 






jtnp 


SHORT $exit$3 






push 


OFFSET $SG2936 ; 


1 skip me ! 1 




call 


_printf 






add 


esp, 4 




$exit$3 


push 


OFFSET $SG2937 ; 


'end' 




call 


_printf 






add 


esp, 4 






xor 


eax, eax 






pop 


ebp 






ret 


0 




_main 


ENDP 







The goto statement has been simpLy repLaced by a JMP instruction, which has the same effect: unconditional jump to another 
place. 

The second printf ( ) could be executed onLy with human intervention, by using a debugger or by patching the code. 
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The second printf ( ) caLL is aLso called “dead code” in compiler terms. This means that the code will never be executed. 
So when you compile this example with optimizations, the compiler removes “dead code”, leaving no trace of it: 



Listing 10.2: Optimizing MSVC 2012 



SSG2981 


DB 


' begin ' , OaH, 00H 




SSG2983 


DB 


' skip me ! ' , OaH, 


00H 


SSG2984 


DB 


’end’, OaH, 00H 




_main 


PROC 








push 


OFFSET $SG2981 ; 


' begin ' 




call 


_printf 






push 


OFFSET $SG2984 ; 


’end’ 


$exit$4 


call 


_printf 






add 


esp, 8 






xor 


eax, eax 






ret 


0 




_main 


ENDP 







However, the compiler forgot to remove the “skip me!” string. 
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Chapter 11 

Conditional jumps 

11.1 Simple example 



#include <stdio.h> 

void f_signed (int a, int b) 

{ 

if (a>b) 

printf ( "a>b\n" ) ; 
if (a==b) 

printf ( "a==b\n" ) ; 
if (a<b) 

printf ( "a<b\n" ) ; 



void f_unsigned (unsigned int a, unsigned int b) 

{ 

if (a>b) 

printf ( "a>b\n" ) ; 
if (a==b) 

printf ( "a==b\n" ) ; 
if (a<b) 

printf ( "a<b\n" ) ; 



int main() 

{ 

f_signed(1, 2); 
f_unsigned( 1 , 2); 
return 0; 

}; 



11.1.1 x86 

x86 + MSVC 

Here is how the f_signed( ) function Looks Like: 

Listing 11.1: Non-optimizing MSVC 2010 



a$ = 8 




_b$ = 12 




_f_signed 


PROC 


push 


ebp 


mov 


ebp, esp 


mov 


eax, DWORD PTR _a$[ebp] 


cmp 


eax, DWORD PTR _b$[ebp] 


jle 


SHORT $LN3@f_signed 


push 


OFFSET $SG737 


call 


_printf 


add 


esp, 4 
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$LN3@f_s 


igned : 




mov 


ecx, DWORD PTR _a$[ebp] 




cmp 


ecx, DWORD PTR _b$[ebp] 




jne 


SHORT $LN2@f_signed 




push 


OFFSET $SG739 ; 'a==b' 




call 


_printf 




add 


esp, 4 




$LN2@f_s 


igned : 




mov 


edx, DWORD PTR _a$[ebp] 




cmp 


edx, DWORD PTR _b$[ebp] 




jge 


SHORT $LN4@f_signed 




push 


OFFSET $SG741 ; ’a<b' 




call 


_printf 




add 


esp, 4 




$LN4@f_s 


igned : 




pop 


ebp 




ret 


0 




_f_signed ENDP 





The first instruction, JLE, stands for Jump if Less or Equal. In other words, if the second operand is Larger or equalto the first 
one, the control flow will be passed to the specified in the instruction address or LabeL. If this condition does not trigger 
because the second operand is smaller than the first one, the control flow would not be altered and the first printf ( ) 
would be executed. The second check is JNE '.Jump if Not Equal. The control flow will not change if the operands are equal. 

The third check is JGE '.Jump if Greater or Equal-\ump if the first operand is Larger than the second or if they are equal. So, 
if all three conditional jumps are triggered, none of the printf ( ) calls would be executed whatsoever. This is impossible 
without speciaL intervention. 

Now Let’s take a Look at the f_unsigned( ) function. The f_unsigned( ) function is the same as f_signed( ), with 
the exception that the JBE and JAE instructions are used instead of JLE and JGE, as follows: 



Listing 11.2: GCC 



_a$ = 8 


; size = 4 


_b$ = 12 


; size = 4 


_f_unsigned PROC 


push 


ebp 


mov 


ebp, esp 


mov 


eax, DWORD PTR _a$[ebp] 


cmp 


eax, DWORD PTR _b$[ebp] 


jbe 


SHORT $LN3@f_unsigned 


push 


OFFSET $SG2761 ; 'a>b' 


call 


_printf 


add 


esp, 4 


$LN3@f_uns 


igned : 


mov 


ecx, DWORD PTR _a$[ebp] 


cmp 


ecx, DWORD PTR _b$[ebp] 


jne 


SHORT $LN2@f_unsigned 


push 


OFFSET $SG2763 ; 'a==b' 


call 


_printf 


add 


esp, 4 


$LN2@f_uns 


igned : 


mov 


edx, DWORD PTR _a$[ebp] 


cmp 


edx, DWORD PTR _b$[ebp] 


jae 


SHORT $LN4@f_unsigned 


push 


OFFSET $SG2765 ; 'a<b' 


call 


_printf 


add 


esp, 4 


$LN4@f_uns 


igned : 


pop 


ebp 


ret 


0 


_f_unsigned ENDP 



As aLready mentioned, the branch instructions are different: JBE -Jump if Below or Equal and JAE -Jump if Above or Equal. 
These instructions (JA/JAE/JB/JBE) differ from JG/JGE/JL/JLE in the fact that they work with unsigned numbers. 

See also the section about signed number representations ( 22 on page 105). That is why if we see JG/J L in use instead of 
JA/JB or vice-versa, we can be almost sure that the variables are signed or unsigned, respectively. 

Here is also the main( ) function, where there is nothing much new to us: 
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x86 + MSVC + Hiew 



CHAPTER 11. CONDITIONAL JUMPS 



We can try to patch the executable fiLe in a way that the f_unsigned( ) function would aLways print “a==b”, no matter the 
input vaLues. Here is how it looks in Hiew: 




Figure 11.1: Hiew: f_unsigned( ) function 



Essentially, we need to accomplish three tasks: 

• force the first jump to always trigger; 

• force the second jump to never trigger; 

• force the third jump to aLways trigger. 

Thus we can direct the code flow to aLways pass through the second printf ( ), and output “a==b”. 

Three instructions (or bytes) has to be patched: 

• The first jump becomes JMP, but the jump offset wouLd remain the same. 

• The second jump might be triggered sometimes, but in any case it will jump to the next instruction, because, we set 
the jump offset to 0. In these instructions the jump offset is added to the address for the next instruction. So if the 
offset is 0, the jump will transfer the controL to the next instruction. 

• The third jump we repLace with JMP just as we do with the first one, so it wiLL aLways trigger. 
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Here is the modified code: 



CHAPTER 11. CONDITIONAL JUMPS 




Figure 11.2: Hiew: let’s modify the f_unsigned( ) function 



If we miss to change any of these jumps, then several printf ( ) calls may execute, while we want to execute only one. 



11.2 Calculating absolute value 



A simple function: 



int my_abs (int i) 

{ 

if (i<0) 

return -i; 

else 

return i; 

}; 



11.2.1 Optimizing MSVC 

This is how the code is usually generated: 

Listing 11.4: Optimizing MSVC 2012 x64 

i$ = 8 

my_abs PROC 
; ECX = input 

test ecx, ecx 
; check for sign of input value 
; skip NEG instruction if sign is positive 
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11.3 Ternary conditional operator 



The ternary conditional operator in C/C++ has the foLLowing syntax: 




Here is an example: 




11.3.1 x86 

OLd and non-optimizing compilers generate assembLy code just as if an if /else statement was used: 

Listing 11.5: Non-optimizing MSVC 2008 

$SG746 DB 'it is ten', 00H 

$SG747 DB 'it is not ten', 00H 

tv65 = -4 ; this will be used as a temporary variable 
_a$ = 8 
_f PROC 

push ebp 

mov ebp, esp 

push ecx 

; compare input value with 10 

cmp DWORD PTR _a$ [ebp] , 10 

; jump to $LN3@f if not equal 
jne SHORT $LN3@f 

; store pointer to the string into temporary variable: 

mov DWORD PTR tv65[ebp], OFFSET $SG746 ; 'it is ten' 

; jump to exit 

jmp SHORT $LN4@f 

$LN3@f : 

; store pointer to the string into temporary variable: 

mov DWORD PTR tv65[ebp], OFFSET $SG747 ; 'it is not ten' 

$LN4@f : 

; this is exit, copy pointer to the string from temporary variable to EAX. 
mov eax, DWORD PTR tv65[ebp] 

mov esp, ebp 

pop ebp 

ret 0 

f ENDP 



Listing 11.6: Optimizing MSVC 2008 
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; j ump 


to $LN4@f 


if equal 






je 


SHORT $LN4@f 






mov 


eax, OFFSET $SG793 ; 


'it is not ten' 


$LN4@f : 










ret 


0 




_f 


ENDP 















Newer compilers are more concise: 



Listing 11.7: Optimizing MSVC 2012 x64 



$SG1 355 DB 


' it is ten' , 00H 




$SG1 356 DB 


1 it is not ten 1 , 00H 




a$ = 8 

f PROC 






; load pointers 


to the both strings 




lea 


rdx, OFFSET FLAT :$SG1 355 ; 'it is 


ten' 


lea 


rax, OFFSET FLAT:$SG1356 ; 'it is 


not ten’ 


; compare input 


value with 10 




cmp 


ecx, 10 




; if equal, copy value from RDX ("it is ten") 




; if not, do nothing, pointer to the string "it is not ten" is still in RAX as for now. 


cmove 


rax, rdx 




ret 


0 




f ENDP 







Optimizing GCC 4.8 for x86 aLso uses the CMOVcc instruction, while the non-optimizing GCC 4.8 uses conditional jumps. 



11.3.2 Let’s rewrite it in an if /else way 



const 


char* f 


(int a) 






{ 


if (a = 


= 10) 










return 


"it 


is ten"; 




else 


return 


"it 


is not ten"; 


}; 











Interestingly, optimizing GCC 4.8 for x86 was aLso able to use CMOVcc in this case: 



Listing 11.8: Optimizing GCC 4.8 



. LCO : 
. LC1 : 
f : 



.string "it is ten" 
.string "it is not ten" 



. LFBO : 

; compare input value with 10 

cmp DWORD PTR [esp+4], 10 

mov edx, OFFSET FLAT:.LC1 ; "it is not ten" 

mov eax, OFFSET FLAT:. LCO ; "it is ten" 

; if comparison result is Not Equal, copy EDX value to EAX 
; if not, do nothing 

cmovne eax, edx 
ret 



But the optimizing MSVC 2012 is not that good (yet). 



11.4 Getting minimal and maximal values 

11.4.1 32-bit 
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Listing 11.9: Non-optimizing MSVC 2013 

_a$ = 8 
_b$ = 12 
_my_min PROC 

push ebp 

mov ebp, esp 

mov eax, DWORD PTR _a$[ebp] 

; compare A and B: 

cmp eax, DWORD PTR _b$[ebp] 

; jump, if A is greater or equal to B: 
jge SHORT $LN2@my_min 

; reload A to EAX if otherwise and jump to exit 
mov eax, DWORD PTR _a$[ebp] 

jmp SHORT $LN3@my_min 

jmp SHORT $LN3@my_min ; this is redundant JMP 

$LN2@my_min : 

; return B 

mov eax, DWORD PTR _b$[ebp] 

$LN3@my_min : 

pop ebp 

ret 0 

_my_min ENDP 

_a$ = 8 
_b$ = 12 
_my_max PROC 

push ebp 

mov ebp, esp 

mov eax, DWORD PTR _a$[ebp] 

; compare A and B: 

cmp eax, DWORD PTR _b$[ebp] 

; jump if A is less or equal to B: 
jle SHORT $LN2@my_max 

; reload A to EAX if otherwise and jump to exit 
mov eax, DWORD PTR _a$[ebp] 

jmp SHORT $LN3@my_max 

jmp SHORT $LN3@my_max ; this is redundant JMP 

$LN2@my_max : 

; return B 

mov eax, DWORD PTR _b$[ebp] 

$LN3@my_max : 

pop ebp 

ret 0 

_my_max ENDP 

These two functions differ onLy in the conditional jump instruction: JGE (“Jump if Greater or EquaL”) is used in the first one 
and JLE (“Jump if Less or EquaL”) in the second. 

There is one unneeded JMP instruction in each function, which MSVC probabLy left by mistake. 
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11.5 Conclusion 

11.5.1 x86 

Here’s the rough skeleton of a conditional jump: 



Listing 11.10: x86 

CMP register, register/value 
Jcc true ; cc=condition code 
false : 

. . . some code to be executed if comparison result is false . . . 
JMP exit 
true : 

. . . some code to be executed if comparison result is true . . . 
exit : 



11.5.2 Branchless 

If the body of a condition statement is very short, the conditional move instruction can be used: MOVcc in ARM (in ARM 
mode), CSEL in ARM64, CMOVcc in x86. 
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Chapter 12 

switch()/case/default 



12.1 Small number of cases 



#include <stdio.h> 

void f (int a) 

{ 

switch (a) 

{ 

case 0: printf ("zero\n"); break; 
case 1: printf ("one\n"); break; 
case 2: printf ("two\n"); break; 
default: printf ("something unknown\n"); break; 
} ; 

}; 

int main() 

{ 

f (2); // test 

}; 



12.1.1 x86 

Non-optimizing MSVC 

Result (MSVC 2010): 



Listing 12.1: MSVC 2010 

tv64 = -4 ; size = 4 
_a$ = 8 ; size = 4 

_f PR0C 

push ebp 

mov ebp, esp 

push ecx 

mov eax, DWORD PTR _a$[ebp] 
mov DWORD PTR tv64[ebp], eax 

cmp DWORD PTR tv64[ebp], 0 

je SHORT $LN4@f 

cmp DWORD PTR tv64[ebp], 1 

je SHORT $LN3@f 

cmp DWORD PTR tv64[ebp], 2 

je SHORT $LN2@f 

jmp SHORT $ LN 1 @f 

$LN4@f : 

push OFFSET $SG739 ; 'zero', OaH, 00H 
call _printf 

add esp, 4 

jmp SHORT $LN7@f 

$LN3@f : 

push OFFSET $SG741 ; 'one', OaH, 00H 
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call 


_printf 




add 


esp, 4 




jmp 


SHORT $LN7@f 




$LN2@f : 


push 


OFFSET $SG743 


'two', OaH, 00H 


call 


_printf 




add 


esp, 4 




jmp 


SHORT $LN7@f 




$ LN 1 @f : 


push 


OFFSET $SG745 


’something unknown’, OaH, 00H 


call 


_printf 




add 


esp, 4 




$LN7@f : 


mov 


esp, ebp 




pop 


ebp 




ret 


0 




_f ENDP 



Our function with a few cases in switch() is in fact analogous to this construction: 

void f (int a) 

{ 

if (a==0) 

printf ("zero\n"); 
else if (a==1 ) 

printf ( "one\n" ) ; 
else if (a==2) 

printf ( "two\n" ) ; 

else 

printf ("something unknown\n"); 



If we work with switchO with a few cases it is impossible to be sure if it was a real switch() in the source code, or just a pack 
of if() statements. This implies that switch() is like syntactic sugar for a Large number of nested if()s. 

There is nothing especially new to us in the generated code, with the exception of the compiler moving input variable a to 
a temporary Local variable tv64 1 . 

If we compile this in GCC 4.4.1, we’ll get almost the same result, even with maximal optimization turned on (-03 option). 



Optimizing MSVC 

Now let’s turn on optimization in MSVC (/Ox): cl l.c /Fal.asm /Ox 

Listing 12.2: MSVC 

_a$ = 8 ; size = 4 
_f PROC 

mov eax, DWORD PTR _a$[esp-4] 

sub eax, 0 

je SHORT $LN4@f 

sub eax, 1 

je SHORT $LN3@f 

sub eax, 1 

je SHORT $LN2@f 



mov 


DWORD PTR 


_a$[esp-4] , 


OFFSET $SG791 ; 


’something unknown’, OaH, 00H 


jmp 
$LN2@f : 


_printf 










mov 


DWORD PTR 


_a$[esp-4] , 


OFFSET $SG789 ; 


' two ' , 


OaH, 00H 


jmp 
$LN3@f : 


_printf 










mov 


DWORD PTR 


_a$[esp-4] , 


OFFSET $SG787 ; 


'one' , 


OaH, 00H 


jmp 
$LN4@f : 


_printf 










mov 


DWORD PTR 


_a$[esp-4] , 


OFFSET $SG785 ; 


’ zero ’ 


, OaH, 00H 


jmp 


_printf 











f ENDP 



1 Local variables in stack are prefixed with tv-that’s how MSVC names internal variables for its needs 
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Here we can see some dirty hacks. 

First: the value of a is placed in EAX and 0 is subtracted from it. Sounds absurd, but it is done to check if the value in EAX 
was 0. If yes, the ZF flag is to be set (e.g. subtracting from 0 is 0) and the first conditional jump JE (Jump ifEquai or synonym 
JZ -Jump if Zero) is to be triggered and controL flow is to be passed to the $LN4@f labeL, where the ' zero ' message is 
being printed. If the first jump doesn’t get triggered, 1 is subtracted from the input value and if at some stage the result is 
0, the corresponding jump is to be triggered. 

And if no jump gets triggered at all, the control flow passes to printf ( ) with string argument ' something unknown ' . 

Second: we see something unusual for us: a string pointer is placed into the a variable, and then printf ( ) is called not via 
CALL, but via JMP. There is a simple explanation for that: the caller pushes a value to the stack and calls our function via 
CALL. CALL itself pushes the return address (RA 2 ) to the stack and does an unconditional jump to our function address. Our 
function at any point of execution (since it do not contain any instruction that moves the stack pointer) has the following 
stack layout: 

• ESP-points to RA 

• ESP+4-points to the a variable 

On the other side, when we need to call printf ( ) here we need exactly the same stack Layout, except for the first printf ( ) 
argument, which needs to point to the string. And that is what our code does. 

It repLaces the function’s first argument with the address of the string and jumps to printf ( ), as if we didn’t call our 
function f ( ), but directly printf ( ). printf ( ) prints a string to stdout and then executes the RET instruction, which 
POPs RA from the stack and control flow is returned not to f ( ) but rather to f ( )’s caLLee, bypassing the end of the f ( ) 
function. 

All this is possible because printf ( ) is caLled right at the end of the f ( ) function in all cases. In some way, it is similar to 
the longjmp( ) 3 function. And of course, it is all done for the sake of speed. 

12.1.2 Conclusion 

A switchO with few cases is indistinguishable from an if/else construction, for example: listing. 12. 1.1. 



12.2 A lot of cases 



If a switch( ) statement contains a Lot of cases, it is not very convenient for the compiler to emit too Large code with a Lot 
JE/JNE instructions. 

#include <stdio.h> 

void f (int a) 

{ 

switch (a) 

{ 

case 0: printf ("zero\n"); break; 

case 1: printf ("one\n"); break; 

case 2: printf ("two\n"); break; 

case 3: printf ("three\n"); break; 

case 4: printf ("four\n"); break; 

default: printf ("something unknown\n"); break; 

}; 

}; 

int main() 

{ 

f (2); // test 

}; 



12.2.1 x86 

Non-optimizing MSVC 

We get (MSVC 2010): 



2 Return Address 

3 wikipedia 
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Listing 12.3: MSVC 2010 


tv64 = -4 


; size 


= 4 


1 

QJ 

-fa^- 

00 


; size 


= 4 


_f PROC 


push 


ebp 




mov 


ebp, esp 


push 


ecx 




mov 


eax, DWORD PTR _a$[ebp] 


mov 


DWORD PTR tv64[ebp], eax 


cmp 


DWORD PTR tv64 [ebp] , 4 


ja 


SHORT $ LN 1 @f 


mov 


ecx, DWORD PTR tv64[ebp] 


jmp 


DWORD PTR $ LN1 1@f [ecx*4] 


$LN6@f : 


push 


OFFSET 


$SG739 ; ’zero’, OaH, OOH 


call 


_printf 


add 


esp, 4 




jmp 


SHORT $LN9@f 


$LN5@f : 


push 


OFFSET 


$SG741 ; ’one’, OaH, OOH 


call 


_printf 


add 


esp, 4 




jmp 


SHORT $LN9@f 


$LN4@f : 


push 


OFFSET 


$SG743 ; ’two’, OaH, OOH 


call 


_printf 


add 


esp, 4 




jmp 


SHORT $LN9@f 


$LN3@f : 


push 


OFFSET 


$SG745 ; ’three’, OaH, OOH 


call 


_printf 


add 


esp, 4 




jmp 


SHORT $LN9@f 


$LN2@f : 


push 


OFFSET 


$SG747 ; 'four', OaH, OOH 


call 


_printf 


add 


esp, 4 




jmp 


SHORT $LN9@f 


$ LN 1 @f : 


push 


OFFSET 


$SG749 ; 'something unknown', OaH, OOH 


call 


_printf 


add 


esp, 4 




$LN9@f : 


mov 


esp, ebp 


pop 


ebp 




ret 


0 




npad 


2 ; align next label 


$ LN 1 1@f : 


DD 


$LN6@f 


0 


DD 


$LN5@f 


1 


DD 


$LN4@f 


2 


DD 


$LN3@f 


3 


DD 


$LN2@f 


4 


_f ENDP 



What we see here is a set of printf ( ) calls with various arguments. All they have not onLy addresses in the memory of 
the process, but aLso internaL symbolic labels assigned by the compiLer. All these Labels are aLso mentioned in the $ LN 1 1@f 
internaL table. 

At the function start, if a is greater than 4, control flow is passed to Label $ LN1 @f , where printf ( ) with argument ' some- 
thing unknown' is caLLed. 

But if the value of a is less or eguaLs to 4, then it gets multiplied by 4 and added with the $ LN 1 1 @f table address. That is 
how an address inside the table is constructed, pointing exactly to the element we need. For example, let’s say a is eguaL to 
2. 2 * 4 = 8 (all table elements are addresses in a 32-bit process and that is why all elements are 4 bytes wide). The address 
of the $ LN 1 1@f table + 8 is the table element where the $LN4@f Label is stored. JMP fetches the $LN4@f address from 
the table and jumps to it. 

This tabLe is sometimes caLLed jumptabie or branch table 4 . 

4 The whole method was once called computed GOTO in early versions of FORTRAN: Wikipedia. Not quite relevant these days, but what a term! 
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Then the corresponding printf( ) is called with argument 'two'. Literally, the jmp DWORD PTR $ LN 1 1 @f [ecx*4] 
instruction implies jump to the DWORD that is stored at address $ LN 1 1 @f + ecx * 4. 

is assembLy Language macro that aLigning the next Label so that it is to be stored at an address aligned on a 4 byte (or 16 
byte) boundary. This is very suitable for the processor since it is able to fetch 32-bit values from memory through the memory 
bus, cache memory, etc, in a more effective way if it is aLigned. 



Non-optimizing GCC 

Let’s see what GCC 4.4.1 generates: 



Listing 12.4: GCC 4.4.1 



public 


f 




f proc near ; CODE XREF: main+10 




var_18 = dword 


ptr - 1 8h 




arg_0 = dword 


ptr 8 




push 


ebp 




mov 


ebp, esp 




sub 


esp, 1 8 h 




cmp 


[ebp+arg_0] , 4 




ja 


short loc_8048444 




mov 


eax, [ebp+arg_0] 




shl 


eax, 2 




mov 


eax, ds :off_804855C[eax] 




jmp 


eax 




loc_80483FE : ; 


DATA XREF: . rodata : of f_804855C 




mov 


[esp+1 8h+var_1 8] , offset aZero ; 


"zero" 


call 


_puts 




jmp 


short locret_8048450 




loc_804840C : ; 


DATA XREF: . rodata : 08048560 




mov 


[esp+1 8h+var_1 8] , offset aOne ; 


"one" 


call 


_puts 




jmp 


short locret_8048450 




loc_804841 A : ; 


DATA XREF: . rodata : 08048564 




mov 


[esp+1 8h+var_1 8] , offset aTwo ; 


"two" 


call 


_puts 




jmp 


short locret_8048450 




loc_8048428 : ; 


DATA XREF: . rodata : 08048568 




mov 


[esp+1 8h+var_1 8] , offset aThree 


; "three" 


call 


_puts 




jmp 


short locret_8048450 




loc_8048436 : ; 


DATA XREF: . rodata : 0804856C 




mov 


[esp+1 8h+var_1 8] , offset aFour ; 


"four" 


call 


_puts 




jmp 


short locret_8048450 




loc_8048444 : ; 


CODE XREF: f+A 




mov 


[esp+1 8h+var_1 8] , offset aSomethingUnkno ; "something unknown" 


call 


_puts 




locret_8048450 

leave 
retn 
f endp 


: ; CODE XREF: f+26 
; f +34 . . . 




of f_804855C dd 


offset loc_80483FE ; DATA XREF 


: f+12 


dd 


offset loc_804840C 




dd 


offset loc_804841A 




dd 


offset loc_8048428 




dd 


offset loc_8048436 
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It is almost the same, with a LittLe nuance: argument arg_0 is multiplied by 4 by shifting it to Left by 2 bits (it is almost 
the same as multiplication by 4) ( 15.2.1 on page 61). Then the address of the label is taken from the of f_804855C array, 
stored in EAX, and then JMP EAX does the actuaL jump. 



12.2.2 Conclusion 

Rough skeLeton of switchO'. 



Listing 12.5: x86 

MOV REG, input 

CMP REG, 4 ; maximal number of cases 
JA default 

SHL REG, 2 ; find element in table, shift for 3 bits in x64. 
MOV REG, jump_table[REG] 

JMP REG 

easel : 

; do something 
JMP exit 
case2 : 

; do something 
JMP exit 
case3 : 

; do something 
JMP exit 
case4 : 

; do something 
JMP exit 
case5 : 

; do something 
JMP exit 

default : 



exit : 



jump_table dd easel 
dd case2 
dd case3 
dd case4 
dd case5 



The jump to the address in the jump table may also be implemented using this instruction: JMP jump_table[REG*4]. 
Or JMP jump_table[REG*8] in x64. 

A jumptable is just array of pointers, like the one described Later: 16.4 on page 68. 



12.3 When there are several case statements in one block 



Here is a very widespread construction: several case statements for a single bLock: 
#include <stdio.h> 

void f(int a) 

{ 

switch (a) 

{ 

case 1 : 
case 2: 
case 7: 
case 10: 

printf ("1 , 2, 7, 10\n"); 
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It’s too wasteful to generate a block for each possible case, so what is usuaLLy done is to generate each block pLus some kind 
of dispatcher. 



12.3.1 MSVC 



1 $SG2798 DB 

2 SSG2800 DB 

3 SSG2802 DB 

4 SSG2804 DB 

5 SSG2806 DB 

6 

7 _a$ = 8 

8 _f PROC 

9 mov 

10 dec 

11 cmp 

12 ja 

13 movzx 

14 jmp 

15 $LN5@f : 

16 mov 

17 jmp 

18 $LN4@f : 

19 mov 

20 jmp 

21 $LN3@f : 

22 mov 

23 jmp 

24 $LN2@f: 

25 mov 

26 jmp 

27 $ LN 1 @f : 

28 mov 

29 jmp 

30 npad 

31 $ LN 1 1 @f : 

32 DD 

33 DD 

34 DD 

35 DD 



Listing 12.6: Optimizing MSVC 2010 

' 1 , 2, 7, 10' , OaH, 00H 
'3, 4, 5' , OaH, 00H 
’8, 9, 21', OaH, 00H 
'22' , OaH, 00H 
’ default ’ , OaH, 00H 



eax, DWORD PTR _a$[esp-4] 
eax 

eax, 21 
SHORT $ L N 1 @f 

eax, BYTE PTR $LN1 0@f [eax] 

DWORD PTR $LN1 1@f [eax*4] 

DWORD PTR _a$ [esp-4] , OFFSET SSG2798 ; '1, 2, 7, 10' 
DWORD PTR imp printf 

DWORD PTR _a$ [esp-4] , OFFSET SSG2800 ; '3, 4, 5' 
DWORD PTR imp printf 

DWORD PTR _a$ [esp-4] , OFFSET SSG2802 ; '8, 9, 21' 
DWORD PTR imp printf 

DWORD PTR _a$ [esp-4], OFFSET $SG2804 ; '22' 

DWORD PTR imp printf 

DWORD PTR _a$ [esp-4] , OFFSET $SG2806 ; 'default' 
DWORD PTR imp printf 

2 ; align $ LN 1 1 @f table on 16-byte boundary 

$LN5@f ; print '1, 2, 7, 10' 

$LN4@f ; print '3, 4, 5' 

$LN3@f ; print '8, 9, 21' 

$LN2@f ; print '22' 
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DD 


$ LN1 @f ; print 'default' 


$LN1 0@f 


DB 


0 


a=1 




DB 


0 


3 = 2 




DB 


1 


3 = 3 




DB 


1 


3=4 




DB 


1 


a=5 




DB 


1 


3 = 6 




DB 


0 


3 = 1 




DB 


2 


a=8 




DB 


2 


a=9 




DB 


0 


a=1 0 




DB 


4 


a=1 1 




DB 


4 


a=1 2 




DB 


4 


a=13 




DB 


4 


a=14 




DB 


4 


a=1 5 




DB 


4 


a=1 6 




DB 


4 


a=1 7 




DB 


4 


a=1 8 




DB 


4 


a=1 9 




DB 


2 


a=20 




DB 


2 


a=21 




DB 


3 


a=22 


f 


ENDP 















We see two tables here: the first table ($ LN 1 0@f) is an index table, and the second one ($ LN 1 1 @f) is an array of pointers to 
bLocks. 

First, the input value is used as an index in the index table (line 13). 

Here is a short Legend for the vaLues in the table: 0 is the first case bLock (for values 1, 2, 7, 10), 1 is the second one (for values 
3, 4, 5), 2 is the third one (for vaLues 8, 9, 21), 3 is the fourth one (for vaLue 22), 4 is for the default block. 

There we get an index for the second table of code pointers and we jump to it (line 14). 

What is also worth noting is that there is no case for input value 0. That’s why we see the DEC instruction at Line 10, and 
the table starts at a = 1 , because there is no need to allocate a table element for a = 0. 

This is a very widespread pattern. 

So why is this economical? Why isn’t it possible to make it as before ( 12.2.1 on page 47), just with one table consisting of 
bLock pointers? The reason is that the elements in index table are 8-bit, hence it’s all more compact. 



12.4 Fall-through 

Another very popular usage of switch( ) is the fall-through. Here is a small example: 

#define R 1 
#define W 2 
#define RW 3 

void f(int type) 

{ 

int read=0, write=0; 

switch (type) 

{ 

case RW: 

read=1 ; 

case W: 

write=1 ; 
break; 

case R: 

read=1 ; 
break; 

default : 

break; 

}; 

printf ("read=%d, write=%d\n", read, write); 
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If type = 1 (R), read is to be set to 1, if type = 2 (W), write is to be set to 2. In case of type = 3 (RW), both read and write is 
to be set to 1. 

The code at Line 14 is executed in two cases: if type = RW or if type = W. There is no “break” for “case RW”x and that’s OK. 



12.4.1 MSVC x86 



Listing 12.7: MSVC 2012 



$SG1 305 


DB 


'read=%d, write=%d ' , OaH, 00H 


_write$ 


= -12 


; size = 4 


_read$ 


= -8 


; size = 4 


tv64 = 


-4 


; size = 4 


type$ 


= 8 


; size = 4 


_f 


PROC 






push 


ebp 




mov 


ebp, esp 




sub 


esp, 12 




mov 


DWORD PTR _read$ [ebp] , 0 




mov 


DWORD PTR _write$ [ebp] , 0 




mov 


eax, DWORD PTR _type$[ebp] 




mov 


DWORD PTR tv64[ebp], eax 




cmp 


DWORD PTR tv64[ebp] , 1 ; R 




je 


SHORT $LN2@f 




cmp 


DWORD PTR tv64[ebp] , 2 ; W 




je 


SHORT $LN3@f 




cmp 


DWORD PTR tv64[ebp], 3 ; RW 




je 


SHORT $LN4@f 




jmp 


SHORT $LN5@f 


$LN4@f : 


; case 


RW: 




mov 


DWORD PTR _read$ [ebp] , 1 


$LN3@f : 


; case 


W: 




mov 


DWORD PTR _write$ [ebp] , 1 




jmp 


SHORT $LN5@f 


$LN2@f : 


; case 


R: 




mov 


DWORD PTR _read$ [ebp] , 1 


$LN5@f : 


; default 




mov 


ecx, DWORD PTR _write$[ebp] 




push 


ecx 




mov 


edx, DWORD PTR _read$[ebp] 




push 


edx 




push 


OFFSET SSG1305 ; 'read=%d, wr 




call 


_printf 




add 


esp, 12 




mov 


esp, ebp 




pop 


ebp 




ret 


0 


f 


ENDP 





The code mostLy resembles what is in the source. There are no jumps between Labels $LN4@f and $LN3@f : so when code 
flow is at $LN4@f, read is first set to 1, then write. This is why it’s caLled faLL-through: code flow faLLs through one piece of 
code (setting read) to another (setting write). If type = W, we land at $LN3@f, so no code setting read to 1 is executed. 
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Chapter 13 

Loops 



13.1 Simple example 

13.1.1 x86 

There is a special. LOOP instruction in x86 instruction set for checking the vaLue in register ECX and if it is not 0, to decrement 
ECX and pass controL flow to the label in the LOOP operand. ProbabLy this instruction is not very convenient, and there are 
no any modern compilers which emit it automatically. So, if you see this instruction somewhere in code, it is most LikeLy that 
this is a manuaLLy written piece of assembly code. 

In C/C++ Loops are usually constructed using for ( ), while( ) or do/while( ) statements. 

Let’s start with for ( ). 

This statement defines Loop initialization (set Loop counter to initia L value), Loop condition (is the counter bigger than a limit?), 
what is done at each iteration (increment/decrement) and of course Loop body. 

for (initialization; condition; at each iteration) 

{ 

loop_body ; 

} 



The generated code is consisting of four parts as well. 
Let’s start with a simple example: 

#include <stdio.h> 

void printing_function(int i) 

{ 

printf ("f(%d)\n" , i); 

}; 

int main() 

{ 

int i; 

for (i=2; i < 1 0 ; i++) 

printing_function( i) ; 

return 0; 

}; 



Result (MSVC 2010): 



Listing 13.1: MSVC 2010 

i$ = -4 
main PR0C 
push ebp 

mov ebp, esp 

push ecx 

mov DWORD PTR _i$[ebp], 2 ; loop initialization 

jmp SHORT $LN3@main 
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$LN2@main 








mov 


eax, DWORD PTR _i$[ebp] 


here is what we do after each iteration 




add 


eax, 1 


add 1 to (i) value 




mov 


DWORD PTR _i$[ebp], eax 






$LN3@main 








cmp 


DWORD PTR _i$ [ebp] , 10 


this condition is checked *before* each 


iteration 


jge 


SHORT $LN1@main 


if (i) is biggest or equals to 10, lets 


finish loop' 


mov 


ecx, DWORD PTR _i$ [ebp] 


loop body: call printing_function(i) 




push 


ecx 






call 


_printing_f unction 






add 


esp, 4 






jmp 


SHORT $LN2@main 


jump to loop begin 




$LN1@main 




loop end 




xor 


eax, eax 






mov 


esp, ebp 






pop 


ebp 






ret 


0 






_main 


ENDP 







As we see, nothing special. 



Listing 13.2: Optimizing MSVC 



_main 


PROC 


push 


esi 


mov 


esi, 2 


$LL3@main 




push 


esi 


call 


_printing_f unction 


inc 


esi 


add 


esp, 4 


cmp 


esi, 10 ; OOOOOOOaH 


ji 


SHORT $LL3@main 


xor 


eax, eax 


pop 


esi 


ret 


0 


_main 


ENDP 



What happens here is that space for the i variable is not allocated in the LocaL stack anymore, but uses an individual register 
for it, ESI. This is possible in such small functions where there aren’t many LocaL variables. 

One very important thing is that the f ( ) function must not change the value in ESI. Our compiler is sure here. And if the 
compiler decides to use the ESI register in f ( ) too, its value would have to be saved at the function’s proLogue and restored 
at the function’s epilogue, almost Like in our listing: please note PUSH ESI/POP ESI at the function start and end. 



13.1.2 One more thing 

In the generated code we can see: after initializing i, the body of the loop is not to be executed, as the condition for i is 
checked first, and onLy after that Loop body can be executed. And that is correct. Because, if the Loop condition is not met 
at the beginning, the body of the loop must not be executed. This is possible in the following case: 

for (i=0; i<total_entries_to_process ; i++) 
loop_body ; 



If total_entries_to_process is 0, the body of the Loo must not be executed at all. This is why the condition checked before the 
execution. 

However, an optimizing compiler may swap the condition check and Loop body, if it sure that the situation described here is 
not possible (like in the case of our very simple example and KeiL, Xcode (LLVM), MSVC in optimization mode). 



13.2 Memory blocks copying routine 



ReaL-worLd memory copy routines may copy 4 or 8 bytes at each iteration, use SIMD 1 , vectorization, etc. But for the sake of 
simplicity, this example is the simplest possible. 

1 SingLe instruction, multiple data 
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#include <stdio.h> 

void my_memcpy (unsigned char* dst, unsigned char* src, size_t cnt) 

{ 

size_t i; 

for (i=0; i<cnt; i++) 
dst[i]=src[i] ; 

}; 



13.2.1 Straight-forward implementation 



Listing 13.3: GCC 4.9 x64 optimized for size (-Os) 



my_memcpy : 








; RDI 


= dest 


ination 


addre 


ss 


; RSI 


= source addre 


ss 




; RDX 


= size 


of block 




; initialize 


counter 


(i) 


at 0 




xor 


eax, 


eax 




. L2 : 










; all 


bytes 


copied? 


exit 


then : 




cmp 


rax, 


rdx 






je 


. L5 






; load byte 


at RSI+i 








mov 


cl, 


BYTE 


PTR [rsi+rax] 


; store byte 


at RDI + 


i : 






mov 


BYTE 


PTR 


[rdi+rax] , cl 




inc 


rax 


; i ++ 






jmp 


. L2 






. L5 : 












ret 









13.3 Conclusion 



Rough skeleton of Loop from 2 to 9 inclusive: 



Listing 13.4: x86 



mov [counter] , 
jmp check 


2 ; initialization 


body: 




; loop body 
; do something 


here 


; use counter 


variable in local stack 


add [counter] , 


1 ; increment 


check : 




cmp [counter] , 
jle body 


9 



The increment operation may be represented as 3 instructions in non-optimized code: 



Listing 13.5: x86 



MOV [counter] , 2 ; initialization 
JMP check 



body 



loop body 
do something here 

use counter variable in local stack 



MOV REG, [counter] ; increment 
INC REG 

MOV [counter] , REG 
check : 



CMP [counter] , 9 
JLE body 
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If the body of the Loop is short, a whole register can be dedicated to the counter variable: 

Listing 13.6: x86 

MOV EBX, 2 ; initialization 
JMP check 
body: 

; loop body 
; do something here 

; use counter in EBX, but do not modify it! 

INC EBX ; increment 
check : 

CMP EBX, 9 
JLE body 



Some parts of the Loop may be generated by compiler in different order: 



Listing 13.7: x86 



MOV [counter] , 


2 ; initialization 


JMP label_check 


label_increment : 




ADD [counter] , 


1 ; increment 


label_check: 




CMP [counter] , 
JGE exit 


10 


; loop body 
; do something 


here 


; use counter 


variable in local stack 


JMP label_increment 


exit : 





UsuaLly the condition is checked before Loop body, but the compiler may rearrange it in a way that the condition is checked 
after Loop body. This is done when the compiler is sure that the condition is aLways true on the first iteration, so the body 
of the loop is to be executed at least once: 



Listing 13.8: x86 

MOV REG, 2 ; initialization 
body: 

; loop body 
; do something here 

; use counter in REG, but do not modify it! 

INC REG ; increment 
CMP REG, 10 
JL body 



Using the LOOP instruction. This is rare, compilers are not using it. When you see it, it’s a sign that this piece of code is 
hand-written: 



Listing 13.9: x86 

; count from 10 to 1 
MOV ECX, 10 
body: 

; loop body 
; do something here 

; use counter in ECX, but do not modify it! 

LOOP body 
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Chapter 14 

Simple C-strings processing 



14.1 strlenQ 



Let’s ta Lk about loops one more time. Often, the strlen( ) function 1 is implemented using a while( ) statement. Here is 
how it is done in the MSVC standard libraries: 

int my_strlen (const char * str) 

{ 

const char *eos = str; 
while( *eos++ ) ; 
return( eos - str - 1 ); 

} 

int main() 

{ 

// test 

return my_strlen( "hello !"); 

}; 



14.1.1 x86 



Non-optimizing MSVC 



Let’s compile: 



_eos$ = -4 ; size = 4 

_str$ = 8 ; size = 4 

_strlen PROC 
push ebp 

mov ebp, esp 

push ecx 

mov eax, DWORD PTR _str$[ebp] ; place pointer to string from "str" 

mov DWORD PTR _eos$[ebp], eax ; place it to local variable "eos" 

$LN2@strlen_: 

mov ecx, DWORD PTR _eos$[ebp] ; ECX=eos 

; take 8-bit byte from address in ECX and place it as 32-bit value to EDX with sign extension 



movsx 


edx , BYTE PTR [ecx] 


mov 


eax, DWORD PTR _eos$[ebp] 


add 


eax , 1 


mov 


DWORD PTR _eos$[ebp], eax 


test 


edx, edx 


je 


SHORT $LN1@strlen_ 


jmp 


SHORT $LN2@strlen_ 



$LN1@strlen_: 



EAX=eos 

increment EAX 

place EAX back to "eos 

EDX is zero? 

yes, then finish loop 

continue loop 



Counting the characters in a string in the C language 
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; here 


we calculate the difference 


between two pointers 


mov 


eax, DWORD PTR _eos$[ebp] 




sub 


eax, DWORD PTR _str$[ebp] 




sub 


eax, 1 


; subtract 1 and return result 


mov 


esp, ebp 




pop 


ebp 




ret 


0 




strlen ENDP 











We get two new instructions here: MOVSX and TEST. 

The first one-MOVSX-takes a byte from an address in memory and stores the value in a 32-bit register. MOVSX stands for 
MOV with Sign-Extend. MOVSX sets the rest of the bits, from the 8th to the 31th, to 1 if the source byte is negative or to 0 if 
is positive. 

And here is why. 

By default, the char type is signed in MSVC and GCC. If we have two values of which one is char and the other is int, ( int 
is signed too), and if the first value contain -2 (coded as OxFE) and we just copy this byte into the int container, it makes 
OxOOOOOOFE, and this from the point of signed int view is 254, but not -2. In signed int, -2 is coded as OxFFFFFFFE. So if 
we need to transfer OxFE from a variable of char type to int, we need to identify its sign and extend it. That is what MOVSX 
does. 

You can aLso read about it in “Signed number representations" section ( 22 on page 105). 

It’s hard to say if the compiler needs to store a char variable in EDX, it could just take a 8-bit register part (for example DL). 
Apparently, the compiler’s register allocator works like that. 

Then we see TEST EDX, EDX. You can read more about the TEST instruction in the section about bit fields ( 17 on page 75). 
Flere this instruction just checks if the vaLue in EDX eguals to 0. 



Optimizing MSVC 

Now let’s compile all this in MSVC 2012, with optimizations turned on (/Ox): 



Listing 14.1: Optimizing MSVC 2012 /ObO 



_str$ = 8 


; size = 4 




_strlen PROC 






mov 


edx, DWORD PTR _str$[esp-4] 


EDX -> pointer to the string 


mov 


eax, edx 


move to EAX 


$LL2@strlen : 






mov 


cl, BYTE PTR [eax] 


CL = *EAX 


inc 


eax 


EAX++ 


test 


cl , cl 


CL==0? 


jne 


SHORT $LL2@strlen 


no, continue loop 


sub 


eax, edx 


calculate pointers difference 


dec 


eax 


decrement EAX 


ret 


0 




_strlen ENDP 






Now it is all simpler. Needless to say, the compiler could use registers with such efficiency only in small functions with a few 
Local variables. 



INC/DEC- are increment/decrement instructions, in other words: add or substract 1 to/from a variable. 
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Chapter 15 

Replacing arithmetic instructions to other ones 



In the pursuit of optimization, one instruction may be replaced by another, or even with a group of instructions. For example, 
ADD and SUB can repLace each other: line 18 in listing.??. 



15.1 Multiplication 

15.1.1 Multiplication using addition 

Here is a simple example: 

Listing 15.1: Optimizing MSVC 2010 

unsigned int f(unsigned int a) 

{ 

return a*8; 

}; 



Multiplication by 8 is replaced by 3 addition instructions, which do the same. Apparently, MSVC’s optimizer decided that 
this code can be faster. 



_TEXT 


SEGMENT 








CO 

II 

-w- 

03 

1 








; size = 4 


_f 


PROC 








; File 


c : \polygon\c\2 


. c 






mov 


eax, 


DWORD PTR _a$ [esp-4] 






add 


eax, 


eax 






add 


eax, 


eax 






add 


eax, 


eax 






ret 


0 






_f 


ENDP 








_TEXT 


ENDS 








END 











15.1.2 Multiplication using shifting 

Multiplication and division instructions by a numbers that’s a power of 2 are often replaced by shift instructions. 

unsigned int f(unsigned int a) 

{ 

return a*4; 

}; 



Listing 15.2: Non-optimizing MSVC 2010 

; size = 4 
ebp 

ebp, esp 

eax, DWORD PTR _a$[ebp] 



a$ = 8 
f PROC 

push 
mov 
mov 
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shl 


eax, 2 




pop 


ebp 




ret 


0 


_f 


ENDP 











Multiplication by 4 is just shifting the number to the left by 2 bits and inserting 2 zero bits at the right (as the Last two bits). 
It is just Like multiplying 3 by 100 -we need to just add two zeroes at the right. 



That’s how the shift left instruction works: 




The added bits at right are aLways zeroes. 



15.1.3 Multiplication using shifting, subtracting, and adding 

It’s still possible to get rid of the multiplication operation when you multiply by numbers like 7 or 17 again by using shifting. 
The mathematics used here is relatively easy. 

32-bit 



#include <stdint.h> 

int f1(int a) 

{ 

return a*7; 

}; 

int f2(int a) 

{ 

return a*28; 

}; 

int f3(int a) 

{ 

return a*17; 

}; 



x86 



Listing 15.3: Optimizing MSVC 2012 

; a *7 
_a$ = 8 



fl 


PROC 










mov 


ecx, 


DWORD 


PTR _a$ [esp-4] 


ECX=a 












lea 


eax, 


DWORD 


PTR [ecx*8] 


EAX=ECX*8 










sub 


eax, 


ecx 




EAX=EAX- ECX= 


ECX*8- 


ECX=ECX 


i*7=a*7 




ret 


0 






fl 


ENDP 








a*28 










a$ = 8 










f 2 


PROC 










mov 


ecx, 


DWORD 


PTR _a$ [esp-4] 


ECX=a 












lea 


eax, 


DWORD 


PTR [ecx*8] 


EAX=ECX*8 
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sub eax, ecx 

; EAX=EAX- ECX=ECX*8- ECX=ECX*7=a*7 
shl eax, 2 

; EAX=EAX«2=(a*7)*4=a*28 
ret 0 

_f 2 ENDP 

; a*1 7 
_a$ = 8 
_f3 PROC 

mov eax, DWORD PTR _a$[esp-4] 

; EAX=a 

shl eax, 4 
; EAX=EAX«4=EAX*16=a*16 

add eax, DWORD PTR _a$[esp-4] 

; EAX=EAX+a=a*1 6+a=a*1 7 
ret 0 

f3 ENDP 



64-bit 

#include <stdint.h> 

int64_t f1(int64_t a) 

{ 

return a*7; 

}; 

int64_t f2(int64_t a) 

{ 

return a*28; 

>; 

int64_t f3(int64_t a) 

{ 

return a*17; 

}; 



x64 



Listing 15.4: Optimizing MSVC 2012 

; a *7 
fl : 

lea rax, [0+rdi*8] 

; RAX=RDI*8=a*8 

sub rax, rdi 

; RAX=RAX- RDI=a*8-a=a*7 
ret 

; a*28 
f2 : 

lea rax, [0+rdi*4] 

; RAX=RDI*4=a*4 

sal rdi, 5 

; RDI=RDI«5=RDI*32=a*32 
sub rdi, rax 

; RDI=RDI- RAX=a*32-a*4=a*28 
mov rax, rdi 

ret 

; a*1 7 
f 3 : 

mov rax, rdi 

sal rax, 4 

; RAX=RAX<<4=a*1 6 
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add rax 
; RAX=a*1 6+a=a*1 7 
ret 



rdi 



15.2 Division 



15.2.1 Division using shifts 

ExampLe of division by 4: 

unsigned int f(unsigned int a) 
{ 

return a/4; 

}; 



We get (MSVC 2010): 



Listing 15.5: MSVC 2010 


a$ 


= 8 






; size = 4 


_f 


PR0C 










mov 


eax, 


DWORD PTR 


_a$ [esp-4] 




shr 


eax, 


2 






ret 


0 






_f 


ENDP 









The SHR ( SHift Right) instruction in this example is shifting a number by 2 bits to the right. The two freed bits at left (e.g., 
two most significant bits) are set to zero. The two least significant bits are dropped. In fact, these two dropped bits are the 
division operation remainder. 



The SHR instruction works just like SHL, but in the other direction. 




It is easy to understand if you imagine the number 23 in the decimal numeraL system. 23 can be easily divided by 10 just by 
dropping last digit (3-division remainder). 2 is left after the operation as a guotient. 



So the remainder is dropped, but that’s OK, we work on integer values anyway, these are not a reaL numbers! 
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Chapter 16 

Arrays 



An array is just a set of variables in memory that lie next to each other and that have the same type 1 . 



16.1 Simple example 



#include <stdio.h> 

int main() 

{ 

int a [20] ; 
int i; 

for (i=0; i<20; i++) 
a[i]=i*2; 

for (i=0; i<20; i++) 

printf ( "a [%d] =%d\n" , i, a[i]); 

return 0; 

}; 



16.1.1 x86 



MSVC 

Let’s compile 




Listing 16. 


_TEXT SEGMENT 




_i$ = -84 


* 


size = 4 


_a$ = -80 


* 


size = 80 


_main 


PROC 




push 


ebp 




mov 


ebp, esp 




sub 


esp, 84 ; 00000054H 


mov 


DWORD PTR _i$ [ebp] , 0 




jmp 


SHORT $LN6@main 




$LN5@main : 


mov 


eax, DWORD PTR _i$[ebp] 




add 


eax, 1 




mov 


DWORD PTR _i$[ebp], eax 




$LN6@main : 


cmp 


DWORD PTR _i$ [ebp] , 20 


; 00000014H 


jge 


SHORT $LN4@main 




mov 


ecx, DWORD PTR _i$[ebp] 




shl 


ecx, 1 





1 AKA 2 “homogeneous container” 
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mov 

mov 

J m P 

$LN4@main : 
mov 
jmp 

$LN2@main : 
mov 
add 
mov 

$LN3@main : 
cmp 
jge 
mov 
mov 
push 
mov 
push 
push 
call 
add 
jmp 

$LN1@main : 
xor 
mov 
pop 
ret 
main 



edx, DWORD PTR _i$[ebp] 

DWORD PTR _a$ [ebp+edx*4] , ecx 
SHORT $LN5@main 

DWORD PTR _i$ [ebp] , 0 
SHORT $LN3@main 

eax, DWORD PTR _i$ [ ebp ] 
eax, 1 

DWORD PTR _i$[ebp], eax 

DWORD PTR _i$ [ebp] , 20 ; 00000014H 

SHORT $LN1@main 

ecx, DWORD PTR _i$[ebp] 

edx, DWORD PTR _a$ [ebp+ecx*4] 

edx 

eax, DWORD PTR _i$[ebp] 
eax 

OFFSET $SG2463 
_printf 

esp, 12 ; OOOOOOOcH 

SHORT $LN2@main 

eax, eax 
esp, ebp 
ebp 
0 

ENDP 



Nothing very special, just two Loops: the first is a filling Loop and second is a printing Loop. The shl ecx , 1 instruction is 
used for value multiplication by 2 in ECX, more about below 15.2.1 on page 61. 

80 bytes are allocated on the stack for the array, 20 elements of 4 bytes. 



16.2 Buffer overflow 



16.2.1 Reading outside array bounds 

So, array indexing is just array[index]. If you study the generated code cLosely, you’ll probably note the missing index bounds 
checking, which could check if it is less than 20. What if the index is 20 or greater? That’s the one C/C++ feature it is often 
blamed for. 

Here is a code that successfully compiles and works: 

#include <stdio.h> 

int main() 

{ 

int a [20] ; 
int i; 

for ( 1=0 ; i<20; i++) 
a [i] =i*2 ; 

printf ( "a [20] =%d\n" , a [20] ) ; 
return 0; 

}; 



Compilation results (MSVC 2008): 

Listing 16.2: Non-optimizing MSVC 2008 
$SG2474 DB , a[20]=%d', OaH, 00H 

_i$ = -84 ; size = 4 
_a$ = -80 ; size = 80 
main PROC 
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push 


ebp 


mov 


ebp, esp 


sub 


esp, 84 


mov 


DWORD PTR _i$ [ebp] , 0 


jinp 


SHORT $LN3@main 


$LN2@main 




mov 


eax, DWORD PTR _i$[ebp] 


add 


eax, 1 


mov 


DWORD PTR _i$[ebp], eax 


$LN3@main 




cmp 


DWORD PTR _i$ [ebp] , 20 


jge 


SHORT $LN1@main 


mov 


ecx, DWORD PTR _i$[ebp] 


shl 


ecx, 1 


mov 


edx, DWORD PTR _i$[ebp] 


mov 


DWORD PTR _a$ [ebp+edx*4] , ecx 


jinp 


SHORT $LN2@main 


$LN1@main 




mov 


eax, DWORD PTR _a$[ebp+80] 


push 


eax 


push 


OFFSET $SG2474 ; ' a [20] =%d ' 


call 


DWORD PTR imp printf 


add 


esp, 8 


xor 


eax, eax 


mov 


esp, ebp 


pop 


ebp 


ret 


0 


_main 


ENDP 


_TEXT 

END 


ENDS 



The code produced this result: 



m C:\Potygon\r.exe 

a[20]=1638280 



Figure 16.1: OLlyDbg: console output 

It is just something that was lying in the stack near to the array, 80 bytes away from its first element. 



16.2.2 Writing beyond array bounds 

OK, we read some vaLues from the stack illegally, but what if we could write something to it? 
Here is what we have got: 

#include <stdio.h> 

int main() 

{ 

int a [20] ; 
int i; 

for (i=0; i<30; i++) 
a [i] =i ; 

return 0; 

}; 



MSVC 

And what we get: 
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Listing 16.3: Non-optimizing MSVC 2008 



_TEXT 


SEGMENT 






_i$ = 


-84 ; size = 4 






_a$ = 


-80 ; size = 80 






_main 


PROC 






push 


ebp 






mov 


ebp, esp 






sub 


esp, 84 






mov 


DWORD PTR _i$ [ebp] , 0 






jmp 


SHORT $LN3@main 






$LN2@main : 






mov 


eax, DWORD PTR _i$ [ ebp ] 






add 


eax, 1 






mov 


DWORD PTR _i$[ebp], eax 






$LN3@main : 






cmp 


DWORD PTR _i$ [ebp] , 30 ; 


0000001 eH 




jge 


SHORT $LN1@main 






mov 


ecx, DWORD PTR _i$[ebp] 






mov 


edx, DWORD PTR _i$[ebp] 


; that 


instruction is obviously redundant 


mov 


DWORD PTR _a$ [ebp+ecx*4] 


edx ; ECX 


could be used as second operand here instead 


jmp 


SHORT $LN2@main 






$LN1@main : 






xor 


eax, eax 






mov 


esp, ebp 






pop 


ebp 






ret 


0 






main 


ENDP 















The compiled program crashes after running. No wonder. Let’s see where exactLy does it is crash. 



65 





CHAPTER 16. ARRAYS 



CHAPTER 16. ARRAYS 



Let’s load it into OLLyDbg, and trace until all 30 elements are written: 




Figure 16.2: OLLyDbg: after restoring the value of EBP 
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E 



CPU - main thread 



JHJxJ 



Address 


Hen 


dunp 




























00403000 




FF 


FF 


FF 


FF 


FF 


FF 


FF 


FF 


FF 


FF 


FF 


01 


00 


00 


00 


00403010 


79 


F5 


AD 


63 


86 


0A 


52 


9C 


01 


00 


00 


00 


48 


28 


75 


00 


00403020 


A0 


40 


75 


00 


00 


00 


MM 


00 


00 


00 


MM 


00 


00 


MM 


MM 


00 


00403030 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403040 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403050 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403060 


00 


00 


00 


00 


00 


MM 


00 


00 


00 


00 


00 


00 


00 


MM 


00 


m 


00403070 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


UU 


00403080 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


0040308© 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


004030A0 


00 


MM 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


004030B0 


00 


00 


00 


00 


00 


UM 


00 


MM 


00 


UU 


00 


00 


00 


00 


00 


00 


004030C0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


004030D© 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


004030E0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


004030F0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


MM 


00 


MM 


00403100 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403110 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403120 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403130 


00 


00 


00 


00 


00 


LUJ 


MM 


MM 


00 


MM 


00 


00 


m 


MM 


00 


MM 


00403140 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403150 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403160 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403170 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


Mil 


00 


00 


00403180 


00 


M0 


00 


00 


00 


MU 


MM 


MM 


00 


00 


00 


00 


00 


00 


00 


00 


00403190 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403 1A0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403 1B0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403 1C0 


00 


00 


00 


00 


00 


00 


00 


UM 


00 


00 


MM 


00 


00 


00 


00 


MM 


00403 1D0 


00 


00 


00 


00 


00 


MM 


00 


MM 


00 


M0 


00 


0M 


00 


00 


00 


00 


00403 1E0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403 1F0 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00403200 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


00 


UM 


00 


MM 


00 


00 



Registers (FPU) 



EAX 00000000 
ECX 000000 ID 
EDX 000000 ID 
EBX 00000000 
ESP 0018FF4C 
EBP 00000014 
ESI 00000001 
EDI 0040337C u. 0040337C 



EIP 00000015 



ES 002B 32b it 0( FFFFFFFF) 

CS 0023 32b it 0( FFFFFFFF) 

SS 002B 32b it 0( FFFFFFFF) 

DS 002B 32b it ©(FFFFFFFF) 

FS 0053 32b it 7EFDD000C FFF) 
GS 002B 32b it ©(FFFFFFFF) 



C 
P 
A 
Z 
s 

T 
D 

0 0 Last Err 00000000 ERROR_SUCCESS 
EFL 00000246 ( NO, NB, E, BE, NS, PE, GE, LE) 

ST0 empty 0.0 
ST1 empty 0.0 
ST2 empty 0.0 
ST3 empty 0.0 
ST4 empty 0.0 
ST5 empty 0.0 
ST6 empty 0.0 



ASCII (ANSI 



y iHcHGRbO 
aMu 







0018FF50 


00000017 










0013FF54 


00000018 


t 








0018FF58 


00000019 


4 








0013FF5C 


0000001 A 


♦ 








0013FF60 


000000 IB 


4- 








0018FF64 


000000 1C 


1- 








0018FF68 


000000 1 D 


# 








0018FF6C 


00000000 










0013FF70 


0018FF58 


X t 








0018FF74 


CB89B877 










0018FF78 


0018FFC4 


“ t 


Pointer to next SI 






0018FF7C 


0040 15F5 


iS0 


SE handler 






0018FF80 


63EDD451 


Q b 3C 








0018FF84 


00000000 










0018FF88 


0018FF94 


«p t 








0018FF8C 


76F6338A 


K33o 


RETURN to kerne 13; 






0018FF90 


7EFDE000 


P«" 








0018FF94 


0018FFD4 


t 








0018FF98 


77D39F72 




RETURN to ntdl 1.7' 






0018FF9C 


7EFDE000 


pw 








0018FFA0 


7E814B08 


□KB" 








0018FFA4 


uunnnnnn 










0018FFA8 


00000000 










0018FFRC 


7EFDE000 


P* 








0018FFB0 


00000000 










0018FFB4 


00000000 










0018FFB3 


00000000 










0018FFBC 


0018FFA0 


a t 








0018FFC0 


00000000 










0018FFC4 


0018FFE4 


t 


Pointer to next SI 






0018FFC8 


77D771F5 


ioti-u 


SE handler 






0018FFCC 


094B718C 


MqKo 






▼ 


0018FFD0 


00000000 



















Figure 16.3: OLLyDbg: EIP was restored, but OLLyDbg can’t disassemble at 0x15 



Now please keep your eyes on the registers. 

EIP is 0x15 now. It is not a legaL address for code-at least for Win32 code! We got there somehow against our will. It is 
aLso interesting that the EBP register contain 0x14, ECX and EDX-OxlD. 

Let’s study stack layout a bit more. 

After the control flow was passed to main( ), the vaLue in the EBP register was saved on the stack. Then, 84 bytes were 
allocated for the array and the i variable. That’s (20+1 )*sizeof (int). ESP now points to the _i variable in the Local 
stack and after the execution of the next PUSH something, something is appearing next to _i. 

That’s the stack Layout while the control is in main( ): 



ESP 


4 bytes allocated for i variable 


ESP+4 


80 bytes allocated for a [20] array 


ESP+84 


saved EBP value 


ESP+88 


return address 



a [ 1 9] =something statement writes the Last int in the bounds of the array (in bounds so far!) 
a [20] =something statement writes something to the place where the vaLue of EBP is saved. 

Please take a Look at the register state at the moment of the crash. In our case, 20 was written in the 20th element. At the 
function end, the function epiLogue restores the originaL EBP value. (20 in decimaL is 0x1 4 in hexadecimaL). Then RET gets 
executed, which is effectively equivalent to POP EIP instruction. 

The RET instruction takes the return address from the stack (that is the address in CRT), which was called main( )), and 21 
iss stored there (0x1 5 in hexadecimaL). The CPU traps at address 0x1 5, but there is no executable code there, so exception 
gets raised. 
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